Project

General

Profile

Actions
Copy link

action #160171

closed

[openQA][assets] Access to openQA assets forbidden size:S

Added by waynechen55 about 2 months ago. Updated about 1 month ago.

Status:
Resolved
Priority:
Normal
Assignee:
mkittler
Category:
Regressions/Crashes
Target version:
openQA Project - Ready
Start date:
2024-05-10
Due date:
2024-05-27
% Done:

0%

Estimated time:

Description

Observation

Browse assets on openqa.suse.de is forbidden. Can not see any medium regardless of medium types, including repo, hdd, iso or fixed.

Steps to reproduce

Find jobs referencing this ticket with the help of
https://raw.githubusercontent.com/os-autoinst/scripts/master/openqa-query-for-job-label ,
call openqa-query-for-job-label poo#160171

Impact

Can not determine what is really stored out there on openqa.suse.de

Problem

May have something to do with web server.

Suggestions

  • Check web server settings
  • Check services running

Workaround

n/a

Files

osd_assets_forbidden.png (41 KB) osd_assets_forbidden.png waynechen55, 2024-05-10 05:37

Related issues 1 (0 open — 1 closed)

Related to openQA Project - action #130636: high response times on osd - Try nginx on OSD size:SResolvedmkittler2024-05-17

Actions
Actions
Copy link
 #1

Updated by waynechen55 about 2 months ago

Actions
Copy link
 #2

Updated by okurz about 2 months ago

  • Category set to Regressions/Crashes
  • Target version set to Ready

Is this a regression or has that never worked in before?

Actions
Copy link
 #3

Updated by waynechen55 about 2 months ago

okurz wrote in #note-2:

Is this a regression or has that never worked in before?

Regression. This works very well before.

Actions
Copy link
 #4

Updated by waynechen55 about 2 months ago

  • File deleted (vm_power_off.png)
Actions
Copy link
 #5

Updated by Julie_CAO about 2 months ago

In some cases openqa tests failed in accessing this repo: https://10.145.10.207/tests/14278456#step/installation/4

Actions
Copy link
 #6

Updated by waynechen55 about 2 months ago

This issue causes openQA test run failure https://openqa.suse.de/tests/14286723 @okurz

Actions
Copy link
 #7

Updated by xlai about 2 months ago

  • Priority changed from Normal to Urgent
Actions
Copy link
 #8

Updated by nanzhang about 2 months ago · Edited

The same issue, it caused vmware online upgrade test failure - https://openqa.suse.de/tests/14279862

Actions
Copy link
 #9

Updated by waynechen55 about 2 months ago

waynechen55 wrote in #note-6:

This issue causes openQA test run failure https://openqa.suse.de/tests/14286723 @okurz

I think this failure is caused by incorrect setting of test suite which was triggered mistakenly somehow. But anyway I think this issue should be fixed.

Actions
Copy link
 #10

Updated by ybonatakis about 2 months ago · Edited

If this is used to work before, the easiest mitigation could be to switch to apache. However this endpoint doesnt seem to work on my local instance even with apache

Actions
Copy link
 #11

Updated by okurz about 2 months ago

  • Related to action #130636: high response times on osd - Try nginx on OSD size:S added
Actions
Copy link
 #12

Updated by mkittler about 2 months ago

  • Status changed from New to In Progress
  • Assignee set to mkittler
Actions
Copy link
 #14

Updated by okurz about 2 months ago · Edited

  • Subject changed from [openQA][assets] Access to openQA assets forbidden to [openQA][assets] Access to openQA assets forbidden auto_review:"Download (curl) error for 'http://openqa.suse.de/":retry

https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/1181 merged and deployed.

And called

export host=openqa.suse.de; failed_since="'2024-05-08'" ./openqa-monitor-investigation-candidates | ./openqa-label-known-issues-multi
Actions
Copy link
 #16

Updated by okurz about 2 months ago

  • Description updated (diff)
Actions
Copy link
 #17

Updated by okurz about 2 months ago

  • Subject changed from [openQA][assets] Access to openQA assets forbidden auto_review:"Download (curl) error for 'http://openqa.suse.de/":retry to [openQA][assets] Access to openQA assets forbidden auto_review:"Download.*curl.*error for.*http://openqa.suse.de/":retry
Actions
Copy link
 #18

Updated by okurz about 2 months ago · Edited

Seems like no further jobs were labeled. Rerunning with generalized regex to get rid of potentially problematic '()' and the "'"

Plus

export host=openqa.suse.de; export failed_since="'2024-05-08'"; for i in $(./openqa-monitor-investigation-candidates | sed 's@^[^0-9]*\([0-9]*\) .*$@\1@g'); do openqa-cli api --osd -X post jobs/$i/duplicate ; done

And for jobs that now are labeled with the current ticket openqa-query-for-job-label poo#160171 yields:

14288091|2024-05-13 12:12:04|done|failed|select_modules_and_patterns+registration||grenache-1
14288093|2024-05-13 11:39:27|done|failed|ha_textmode_skip_registration||grenache-1
14287663|2024-05-13 07:33:20|done|incomplete|online_upgrade_sles15sp5_vmware|abandoned: associated worker sapworker2:7 re-connected but abandoned the job|sapworker2
14279413|2024-05-13 07:15:07|done|failed|sev-es-gi-guest_sles15sp5-on-host_developing-kvm||imagetester
14279434|2024-05-12 10:50:16|done|failed|prepare-host_developing-xen||worker35
14281144|2024-05-11 18:36:10|done|failed|SAPHanaSR_ScaleUp_PerfOpt_WMP_node01||worker35
14279360|2024-05-11 17:37:53|done|failed|create_hdd_textmode||worker33
14281003|2024-05-11 17:21:27|done|failed|fips_env_mode_tests_crypt_tool||worker31
14281009|2024-05-11 17:20:21|done|failed|fips_ker_mode_tests_crypt_tool||worker33
14281246|2024-05-11 16:56:09|done|failed|jeos-ltp-commands||worker34
Actions
Copy link
 #19

Updated by okurz about 2 months ago

  • Due date set to 2024-05-27
  • Status changed from In Progress to Feedback
  • Priority changed from Urgent to Normal

this should reduce urgency. I assume you don't plan further implementation so we can monitor. I suggest to review the scenarios linked to the labeled failed jobs before resolving so that this ticket wouldn't be reopened afterwards.

Actions
Copy link
 #20

Updated by mkittler about 2 months ago

sounds good

Actions
Copy link
 #21

Updated by mkittler about 2 months ago · Edited

I checked some scenarios. Two scenarios are still failing:

Maybe #160239 was still causing these tests to fail (considering https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/1188 was merged only 24 minutes ago).

Right now I can download e.g. https://openqa.suse.de/assets/repo/SLE-15-SP6-Full-ppc64le-Build88.1-Media1/Module-Basesystem/ppc64le/keyutils-1.6.3-5.6.1.ppc64le.rpm just fine and listing e.g. https://openqa.suse.de/assets/repo/SLE-15-SP6-Full-ppc64le-Build88.1-Media1 works as well. (So not sure why these tests fails due to /var/log/YaST2/y2log-1.gz:2024-05-13 17:28:13 <5> grenache-7(3908) [zypp-core] Exception.cc(log):202 MediaCurl.cc(getFileCopy):795 RETHROW: Downloaded data exceeded the expected filesize '117.6 KiB' of 'http://openqa.suse.de/assets/repo/SLE-15-SP6-Full-ppc64le-Build88.1-Media1/Module-Basesystem/ppc64le/keyutils-1.6.3-5.6.1.ppc64le.rpm'..)

The majority of scenarios look good now:

Some scenarios don't seem to be relevant:

Actions
Copy link
 #22

Updated by okurz about 2 months ago

mkittler wrote in #note-21:

I checked some scenarios. Two scenarios are still failing:

[…]

Maybe #160239 was still causing these tests to fail (considering https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/1188 was merged only 24 minutes ago).

yes, my mistake.

Actions
Copy link
 #23

Updated by waynechen55 about 2 months ago · Edited

I can access to assets folder on openqa.suse.de now.

Actions
Copy link
 #24

Updated by livdywan about 2 months ago

  • Subject changed from [openQA][assets] Access to openQA assets forbidden auto_review:"Download.*curl.*error for.*http://openqa.suse.de/":retry to [openQA][assets] Access to openQA assets forbidden auto_review:"Download.*curl.*error for.*http://openqa.suse.de/":retry size:s
Actions
Copy link
 #25

Updated by okurz about 2 months ago

  • Subject changed from [openQA][assets] Access to openQA assets forbidden auto_review:"Download.*curl.*error for.*http://openqa.suse.de/":retry size:s to [openQA][assets] Access to openQA assets forbidden auto_review:"Download.*curl.*error for.*http://openqa.suse.de/":retry size:S
Actions
Copy link
 #26

Updated by okurz about 1 month ago 

$ openqa-query-for-job-label poo#160171
14337364|2024-05-17 07:49:45|done|parallel_failed|cc_net_case_client||worker29
14337335|2024-05-17 07:38:38|done|failed|cc_net_case_server||worker-arm1
14336776|2024-05-17 06:21:38|done|parallel_failed|cc_net_case_client_uefi||worker29
14336775|2024-05-17 06:21:37|done|failed|cc_net_case_server_uefi||worker34
14336773|2024-05-17 06:19:41|done|failed|cc_net_case_server||worker31
14336778|2024-05-17 06:18:35|done|failed|cc_net_case_client||worker-arm1
14336781|2024-05-17 06:17:32|done|failed|cc_audit_remote_server||worker-arm2
14336767|2024-05-17 06:14:44|done|failed|cc_audit_remote_client_uefi||worker40
14336766|2024-05-17 06:14:44|done|failed|cc_audit_remote_server_uefi||worker31
14336764|2024-05-17 06:14:35|done|failed|cc_audit_remote_server||worker30
Actions
Copy link
 #27

Updated by mkittler about 1 month ago

  • Subject changed from [openQA][assets] Access to openQA assets forbidden auto_review:"Download.*curl.*error for.*http://openqa.suse.de/":retry size:S to [openQA][assets] Access to openQA assets forbidden size:S
  • Status changed from Feedback to Resolved

None of those jobs are relevant. They failed due to:

Error message: Could not resolve host: openqa.suse.de

That error has a completely different cause. Considering I fixed the serving of assets we can be sure that this issue is no longer the cause of download errors so I'm removing the auto review regex.

At this point too many jobs have been mislabeled to remove labels of relevant jobs manually. So I just ran delete from comments where text like '%poo#160171%'; to delete all comments referencing this issue.

With that I would also consider this ticket resolved. (The scenarios mentioned in #160171#note-21 look still good.)

Actions
Copy link

Also available in: Atom PDF