action #114421
closed
coordination #80142: [saga][epic] Scale out: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes
coordination #92854: [epic] limit overload of openQA webUI by heavy requests
action #97190: Limit size of initial requests everywhere, e.g. /, /tests, etc., over webUI and API
Add a limit where it makes sense after we have it for /tests, query configurable size:M
Added by okurz over 2 years ago.
Updated about 2 years ago.
Category:
Feature requests
Description
Acceptance criteria¶
- AC1: Every query or API route mentioned in #113189 has a reasonable limit
- AC2: There is a way for users to get all the data, e.g. with pagination
- AC3: By default we still show a sensible number
Suggestions¶
- Add limit for easy endpoints, and split out the harder ones (multiple SQL queries, paging...) into separate tickets
- Wait for outcome of #111770
- Ensure there is a query parameter available to users to configure the number of results
- Ensure there is a server-side limit
- Add help text on the web UI (or documentation)
- Watch out for a combination of dbix and perl code filtering/ processing queries
- Allow users to get all data but each query limited
- Optional: UI elements to configure the limit directly
- As an example for introducing a limit, have a look at https://github.com/os-autoinst/openQA/commit/4242bb107e032c27d5f99c91a358adc9640f5ca6
- Copied from action #111770: Limit finished tests on /tests, but query configurable and show complete number of jobs size:S added
- Description updated (diff)
- Status changed from New to Blocked
- Assignee set to okurz
- Status changed from Blocked to Workable
- Assignee deleted (
okurz)
#111770 resolved, work can continue
- Subject changed from Add a limit where it makes sense after we have it for /tests, query configurable to Add a limit where it makes sense after we have it for /tests, query configurable size:M
- Description updated (diff)
- Description updated (diff)
- Assignee set to robert.richardson
- Status changed from Workable to In Progress
Let's assume this is in Progress. Also discussing this in the mob session today.
- Status changed from In Progress to Workable
i'll be gone next week, so i set it back to workable for now
- Status changed from Workable to In Progress
- Copied to action #119428: Ensure users can get all the data for limited queries, e.g. with pagination added
- Copied to action #119431: Inform users e.g. in the webUI if not all results are returned size:M added
As discussed all following routes should use a generic default limit, not have explicitly named ones
All limit related unit tests and implementations seem to work (see the above draft PR), except for t/ui/01-list.t, which passes the new subtests but still fails at the end.
Dubious, test returned 254 (wstat 65024, 0xfe00)
All 34 subtests passed
Test Summary Report
t/ui/01-list.t (Wstat: 65024 (exited 254) Tests: 34 Failed: 0)
Non-zero exit status: 254
Parse errors: No plan found in TAP output
The test fails because job_99927 can't be found.
That is because now there are more than 10 scheduled jobs in the list, so 99927 is moved to the next page.
I think there are various ways to fix that and I would have to have a closer look.
Maybe cancel or delete the jobs you created for this PR after the specific test.
I've lowered the amount of created jobs to 6 (still enough to check the max limit) and also all additional jobs are now deleted when the subtest is done. Now all tests pass, thanks! :)
- Related to action #120315: openqa-client does not get complete asset list size:S added
- Status changed from In Progress to Resolved
There's a follow-up ticket but we assume it's otherwise looking good.
Also available in: Atom
PDF