Project

General

Profile

action #111833

Allow tests/overview page to handle more than 500 jobs with result filter conditions size: M

Added by kraih 2 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
Feature requests
Target version:
Start date:
2022-05-31
Due date:
2022-06-16
% Done:

0%

Estimated time:
Difficulty:
hard

Description

Motivation

Since the patch from #110680 has been applied there are some use cases where the tests/overview page no longer works as expected by our users. This link for example should not be showing the limit warning. But it does because the SQL queries retrieving the jobs (and which apply the limit) do not consider certain filter conditions such as result yet. Those are handled by the unoptimised pure Perl aggregate code in a later step.

Acceptance criteria

  • AC1: This link does not trigger the result limit warning anymore
  • AC2: This link still does trigger the result limit warning

Suggestions

  • Do not revert the patch from #110680, since that would create a DoS vulnerability
  • Try to move the aggregate code into earlier SQL queries
  • Bonus points for moving more filter conditions from Perl to SQL

History

#1 Updated by kraih 2 months ago

Curious that module result is already handled efficiently in SQL, but job result is not.

#2 Updated by kraih 2 months ago

  • Subject changed from Allow tests/overview page to handle more than 500 jobs with result filter conditions to Allow tests/overview page to handle more than 500 jobs with result filter conditions size: M

#3 Updated by kraih 2 months ago

  • Status changed from New to Workable

#4 Updated by kraih 2 months ago

  • Assignee set to kraih

#5 Updated by kraih 2 months ago

  • Status changed from Workable to In Progress

#6 Updated by kraih 2 months ago

Simply moving the condition from Perl to SQL does not appear to work, i'll have to dive a bit deeper into the code.

#7 Updated by openqa_review 2 months ago

  • Due date set to 2022-06-16

Setting due date based on mean cycle time of SUSE QE Tools

#8 Updated by kraih 2 months ago

After running a bunch of experiments i located the problem, simply moving the condition to SQL does not work because of the duplicate removal code here. It depends on the presence of old jobs with different states. The multiple layers of SQL generation and Perl filtering make this a very complex issue. I will have to try to move the duplicate removal to SQL too, or find a different solution for the job limit.

#10 Updated by kraih 2 months ago

  • Status changed from In Progress to Feedback

#11 Updated by kraih about 2 months ago

  • Status changed from Feedback to Resolved

Everything looks good.

Also available in: Atom PDF