Project

General

Profile

action #120315

openqa-client does not get complete asset list size:S

Added by dimstar 3 months ago. Updated 2 months ago.

Status:
Resolved
Priority:
Urgent
Assignee:
okurz
Category:
Concrete Bugs
Target version:
Ready
Start date:
2022-11-11
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Observation

The test queries openQA for the existing assets, using the command:

openqa-client --host https://openqa.opensuse.org assets get | sed -n 's/.name.*Tumbleweed-NET-x86_64-Snapshot\([0-9]\+\)-Media.\$/\1/p'

But since recently (apparently today, the test passed yesterday), ew don't get the complete list anymore:

openqa-client --host https://openqa.opensuse.org assets get | grep -c "name =" => 500

openQA test in scenario opensuse-Tumbleweed-DVD-x86_64-openqa_bootstrap@64bit fails in
start_test

Test suite description

Maintainer: dheidler. Install openQA using openqa-bootstrap script.

Reproducible

Fails since (at least) Build 20221110

Expected result

Last good: 20221109 (or more recent)

Further details

Always latest result in this scenario: latest

Related issues

Related to openQA Project - action #114421: Add a limit where it makes sense after we have it for /tests, query configurable size:MResolved

Related to openQA Project - action #120841: Add pagination for GET /api/v1/assets size:MResolved2022-11-22

Copied to openQA Project - action #120336: Improve scalability of assets listing (API route) size:MWorkable2022-11-11

History

#1 Updated by okurz 3 months ago

  • Project changed from openQA Tests to openQA Project
  • Category changed from Bugs in existing tests to Concrete Bugs
  • Priority changed from Normal to Urgent
  • Target version set to Ready

#2 Updated by okurz 3 months ago

Either the test is adjusted to query more assets or we bump the default limit in openQA assuming that the list of assets isn't too verbose nor expensive

#3 Updated by coolo 3 months ago

It shouldn't 500, but expanding the limit sounds backwards. Supporting a filter sounds what this use case wants.

#4 Updated by coolo 3 months ago

Ah, I thought the HTTP status code is 500. That's an unfortunate choice of default limit :)

#5 Updated by dimstar 3 months ago

okurz wrote:

Either the test is adjusted to query more assets or we bump the default limit in openQA assuming that the list of assets isn't too verbose nor expensive

openqa-client --host https://openqa.opensuse.org assets get limit=100000 | grep -c "name ="
5000

Interestingly, when passing a limit to the get, the number increases but seems to hit a different limiter (the relevant assets are not in the output yet)

#6 Updated by mkittler 3 months ago

  • Assignee set to mkittler

The 2nd limit is so no one can easily overload the server with requests that would run into a gateway timeout anyways. Of course both limits were way too low. I suppose we should have a distinct set of limits for assets that are significantly higher.

#7 Updated by mkittler 3 months ago

I've been changing the default limit to 10000 and you can go up to 50000 with the limit parameter. (The changes are done in /etc/openqa/openqa.ini, you can change it yourself if you need to.)

#8 Updated by cdywan 3 months ago

  • Related to action #114421: Add a limit where it makes sense after we have it for /tests, query configurable size:M added

#9 Updated by mkittler 3 months ago

  • Status changed from New to In Progress

PR to allow configuring this separately for assets: https://github.com/os-autoinst/openQA/pull/4896

I see you're grepping on the whole list to find an asset by its name. I suppose adding a filter/search parameter for that would be helpful. What are you searching for specifically? Is an exact match for the asset name sufficient or do you need globbing?

#10 Updated by mkittler 3 months ago

To answer my last question: The test does a regex match:

 last_tw_build=\$(openqa-client --host $openqa_url assets get | sed -n 's/^.*name.*Tumbleweed-NET-$arch-Snapshot\\([0-9]\\+\\)-Media.*\$/\\1/p' | sort -n | tail -n 1)
 echo "Last Tumbleweed build on openqa.opensuse.org: \$last_tw_build"

This could be easily implemented server-side.

#11 Updated by cdywan 3 months ago

  • Copied to action #120336: Improve scalability of assets listing (API route) size:M added

#12 Updated by cdywan 3 months ago

  • Subject changed from openqa-client does not get complete asset list to openqa-client does not get complete asset list size:S

#13 Updated by mkittler 3 months ago

  • Status changed from In Progress to Resolved

cdywan created a ticket for further improvements so I'll keep it at the mentioned PR for the sake of this ticket. It has already been merged and deployed on o3. I've just reverted my config changes from #120315#note-7 and restarted the service. Now all assets are returned again:

openqa=# select count(*) from assets;
 count 
-------
 15645
(1 Zeile)

$ openqa-cli api --host https://openqa.opensuse.org assets | jq '.assets | length'
15645

#14 Updated by dzedro 3 months ago

Now I can't find tests on osd.
All from 501th will now be not shown ?
https://suse.slack.com/archives/C02CANHLANP/p1668584467225009

#15 Updated by dzedro 3 months ago

Another thing, how many jobs are now scheduled ?
ALso I can't see jobs from 501th on the page.
500 scheduled jobs (85 blocked by other jobs)

#16 Updated by cdywan 3 months ago

dzedro wrote:

Now I can't find tests on osd.
All from 501th will now be not shown ?
https://suse.slack.com/archives/C02CANHLANP/p1668584467225009

As a workaround I added [misc_limits] generic_default_limit = 5000 followed by sudo systemctl restart openqa-webui. The proper solution will be implemented as part of #19428.

#17 Updated by okurz 3 months ago

  • Status changed from Resolved to In Progress
  • Assignee changed from mkittler to okurz

Preparing more limit-bumps to handle those regressions

#19 Updated by okurz 3 months ago

  • Status changed from In Progress to Feedback

#21 Updated by kraih 3 months ago

  • Related to action #120841: Add pagination for GET /api/v1/assets size:M added

#22 Updated by okurz 2 months ago

  • Status changed from Feedback to Resolved

https://github.com/os-autoinst/openQA/pull/4912 merged and deployed to OSD means that we can remove the manual override in /etc/openqa/openqa.ini. I removed the line and restarted the webUI. I can confirm that https://openqa.suse.de/admin/test_suites still shows 2.8k entries so all good.

Also available in: Atom PDF