Project

General

Profile

Actions
Copy link

action #114421

closed

coordination #80142: [saga][epic] Scale out: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes

coordination #92854: [epic] limit overload of openQA webUI by heavy requests

action #97190: Limit size of initial requests everywhere, e.g. /, /tests, etc., over webUI and API

Add a limit where it makes sense after we have it for /tests, query configurable size:M

Added by okurz over 2 years ago. Updated about 2 years ago.

Status:
Resolved
Priority:
Low
Assignee:
robert.richardson
Category:
Feature requests
Target version:
Ready
Start date:
Due date:
% Done:

0%

Estimated time:

Description

Acceptance criteria

  • AC1: Every query or API route mentioned in #113189 has a reasonable limit
  • AC2: There is a way for users to get all the data, e.g. with pagination
  • AC3: By default we still show a sensible number

Suggestions

  • Add limit for easy endpoints, and split out the harder ones (multiple SQL queries, paging...) into separate tickets
  • Wait for outcome of #111770
  • Ensure there is a query parameter available to users to configure the number of results
  • Ensure there is a server-side limit
  • Add help text on the web UI (or documentation)
  • Watch out for a combination of dbix and perl code filtering/ processing queries
  • Allow users to get all data but each query limited
  • Optional: UI elements to configure the limit directly
  • As an example for introducing a limit, have a look at https://github.com/os-autoinst/openQA/commit/4242bb107e032c27d5f99c91a358adc9640f5ca6

Related issues 5 (2 open — 3 closed)

Related to openQA Project (public) - action #113189: Research where we need limits size:SResolvedmkittler2022-07-04

Actions
Related to openQA Project (public) - action #120315: openqa-client does not get complete asset list size:SResolvedokurz2022-11-11

Actions
Copied from openQA Project (public) - action #111770: Limit finished tests on /tests, but query configurable and show complete number of jobs size:SResolvedmkittler2022-05-30

Actions
Copied to openQA Project (public) - action #119428: Ensure users can get all the data for limited queries, e.g. with paginationNew2022-11-22

Actions
Copied to openQA Project (public) - action #119431: Inform users e.g. in the webUI if not all results are returned size:MWorkable2022-10-26

Actions
Actions
Copy link
 #1

Updated by okurz over 2 years ago

  • Copied from action #111770: Limit finished tests on /tests, but query configurable and show complete number of jobs size:S added
Actions
Copy link
 #2

Updated by okurz over 2 years ago

  • Related to action #113189: Research where we need limits size:S added
Actions
Copy link
 #3

Updated by okurz over 2 years ago

  • Description updated (diff)
  • Status changed from New to Blocked
  • Assignee set to okurz
Actions
Copy link
 #4

Updated by okurz over 2 years ago

  • Status changed from Blocked to Workable
  • Assignee deleted (okurz)

#111770 resolved, work can continue

Actions
Copy link
 #5

Updated by livdywan over 2 years ago

  • Subject changed from Add a limit where it makes sense after we have it for /tests, query configurable to Add a limit where it makes sense after we have it for /tests, query configurable size:M
  • Description updated (diff)
Actions
Copy link
 #6

Updated by mkittler over 2 years ago

  • Description updated (diff)
Actions
Copy link
 #7

Updated by mkittler over 2 years ago

  • Assignee set to robert.richardson
Actions
Copy link
 #8

Updated by livdywan about 2 years ago

  • Status changed from Workable to In Progress

Let's assume this is in Progress. Also discussing this in the mob session today.

Actions
Copy link
 #10

Updated by robert.richardson about 2 years ago

  • Status changed from In Progress to Workable

i'll be gone next week, so i set it back to workable for now

Actions
Copy link
 #11

Updated by robert.richardson about 2 years ago

  • Status changed from Workable to In Progress
Actions
Copy link
 #13

Updated by okurz about 2 years ago

  • Copied to action #119428: Ensure users can get all the data for limited queries, e.g. with pagination added
Actions
Copy link
 #14

Updated by okurz about 2 years ago

  • Copied to action #119431: Inform users e.g. in the webUI if not all results are returned size:M added
Actions
Copy link
 #15

Updated by okurz about 2 years ago

As discussed all following routes should use a generic default limit, not have explicitly named ones

Actions
Copy link
 #17

Updated by robert.richardson about 2 years ago

All limit related unit tests and implementations seem to work (see the above draft PR), except for t/ui/01-list.t, which passes the new subtests but still fails at the end.
Dubious, test returned 254 (wstat 65024, 0xfe00)
All 34 subtests passed
Test Summary Report
t/ui/01-list.t (Wstat: 65024 (exited 254) Tests: 34 Failed: 0)
Non-zero exit status: 254
Parse errors: No plan found in TAP output

Actions
Copy link
 #18

Updated by tinita about 2 years ago

The test fails because job_99927 can't be found.
That is because now there are more than 10 scheduled jobs in the list, so 99927 is moved to the next page.
I think there are various ways to fix that and I would have to have a closer look.
Maybe cancel or delete the jobs you created for this PR after the specific test.

Actions
Copy link
 #19

Updated by robert.richardson about 2 years ago

I've lowered the amount of created jobs to 6 (still enough to check the max limit) and also all additional jobs are now deleted when the subtest is done. Now all tests pass, thanks! :)

Actions
Copy link
 #20

Updated by livdywan about 2 years ago

  • Related to action #120315: openqa-client does not get complete asset list size:S added
Actions
Copy link
 #21

Updated by livdywan about 2 years ago

  • Status changed from In Progress to Resolved

There's a follow-up ticket but we assume it's otherwise looking good.

Actions
Copy link

Also available in: Atom PDF