Project

General

Profile

action #110680

coordination #80142: [saga][epic] Scale out: Redundant/load-balancing deployments of openQA, easy containers, containers on kubernetes

coordination #92854: [epic] limit overload of openQA webUI by heavy requests

Overview page shouldn't allow long-running requests without limits size:M

Added by cdywan about 2 months ago. Updated 19 days ago.

Status:
Resolved
Priority:
Normal
Assignee:
kraih
Category:
Feature requests
Target version:
Ready
Start date:
2022-02-03
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Motivation

To resolve #106759#note-37 the endpoint /tests/overview needs a (reasonably high) limit on the number of jobs to prevent heartbeat and reverse proxy timeouts from being triggered.

Acceptance criteria

  • AC1: /tests/overview requests are no longer able to block the prefork worker process for more than 400 seconds
  • AC2: No limits are hard-coded (e.g. the limit can be configured in openqa.ini)

Suggestions

  • Add a hard limit on the number of database rows
  • Render an error message e.g. "Only n results included, please narrow down your search parameters"
  • Use the /tests/overview?version=Tumbleweed&groupid=24 dataset for testing

Related issues

Related to openQA Project - action #97190: Limit size of initial requests everywhere, e.g. /, /tests, etc.New2022-05-30

Blocks openQA Project - action #106759: Worker xyz has no heartbeat (400 seconds), restarting repeatedly reported on o3 size:MFeedback2022-02-03

History

#1 Updated by cdywan about 2 months ago

  • Copied from action #106759: Worker xyz has no heartbeat (400 seconds), restarting repeatedly reported on o3 size:M added

#2 Updated by okurz about 2 months ago

  • Project changed from openQA Infrastructure to openQA Project
  • Due date deleted (2022-05-13)
  • Category set to Feature requests

#3 Updated by okurz about 2 months ago

  • Parent task set to #92854

#4 Updated by kraih about 2 months ago

  • Assignee set to kraih

Been some time since i worked on the frontend.

#5 Updated by cdywan about 2 months ago

  • Copied from deleted (action #106759: Worker xyz has no heartbeat (400 seconds), restarting repeatedly reported on o3 size:M)

#6 Updated by cdywan about 2 months ago

  • Blocked by action #106759: Worker xyz has no heartbeat (400 seconds), restarting repeatedly reported on o3 size:M added

#7 Updated by cdywan about 2 months ago

  • Blocked by deleted (action #106759: Worker xyz has no heartbeat (400 seconds), restarting repeatedly reported on o3 size:M)

#8 Updated by cdywan about 2 months ago

  • Blocks action #106759: Worker xyz has no heartbeat (400 seconds), restarting repeatedly reported on o3 size:M added

#9 Updated by okurz about 2 months ago

  • Related to action #97190: Limit size of initial requests everywhere, e.g. /, /tests, etc. added

#10 Updated by kraih about 1 month ago

  • Status changed from Workable to In Progress

#11 Updated by kraih about 1 month ago

I have replicated the issue locally with a copy of O3.

#12 Updated by openqa_review about 1 month ago

  • Due date set to 2022-06-07

Setting due date based on mean cycle time of SUSE QE Tools

#14 Updated by kraih about 1 month ago

Side note: the actual cause for prefork worker processes blocking everything and getting killed was the query that retrieves the comments for the jobs on the /tests/overview page.

#15 Updated by kraih about 1 month ago

  • Status changed from In Progress to Feedback

PR has been merged.

#16 Updated by kraih 28 days ago

  • Status changed from Feedback to Resolved

#17 Updated by okurz 27 days ago

  • Status changed from Resolved to Feedback

As discussed in https://suse.slack.com/archives/C02CANHLANP/p1653980932015039 https://openqa.opensuse.org/tests/overview?result=incomplete&groupid=1&groupid=3 shows "Only 500 results included, please narrow down your search parameters." even when there are zero jobs displayed

#18 Updated by kraih 27 days ago

The new problem has its own ticket now: https://progress.opensuse.org/issues/111833 #111833

#19 Updated by kraih 19 days ago

  • Status changed from Feedback to Resolved

#20 Updated by okurz 19 days ago

  • Due date deleted (2022-06-07)

Also available in: Atom PDF