Project

General

Profile

Actions

action #100676

open

obs-rsync failed to trigger tests after hitting "Bad gateway", manual retry helped

Added by okurz about 3 years ago. Updated about 3 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Regressions/Crashes
Target version:
Start date:
2021-10-11
Due date:
% Done:

0%

Estimated time:

Description

Observation

From https://suse.slack.com/archives/C02CANHLANP/p1633926701183100

Hi, we are missing tests running on x86_64 & s390 with online flavor on the latest build 47.1:hand: Only arch64 & ppc64 tests got started yesterday
looking. Following https://openqa.suse.de/admin/obs_rsync I can find https://openqa.suse.de/admin/obs_rsync/SUSE:SLE-15-SP4:GA:TEST on where I can see https://openqa.suse.de/admin/obs_rsync/SUSE:SLE-15-SP4:GA:TEST%7Cbase from where I can find all ISO files. The last sync is referencing https://openqa.suse.de/minion/jobs?id=3136300 which is "failed" but not more details
https://openqa.suse.de/admin/obs_rsync/SUSE:SLE-15-SP4:GA:TEST%7Cbase/runs/.run_last/download/openqa.cmd.log aborts with "Bad gateway" so likely it stopped there
the commands are available on https://openqa.suse.de/admin/obs_rsync/SUSE:SLE-15-SP4:GA:TEST%7Cbase/runs/.run_last/download/openqa.cmd . let me execute them manually now
x86_64 jobs are scheduled, s390x ongoing
s390x jobs have been triggered as well

Acceptance criteria

  • AC1: The cause of "Bad gateway is understood and similar situations prevented
  • AC2: obs-rsync retries on "Bad gateway"

Workaround

Execute the trigger tasks that are available as "openqa.cmd" manually

Actions #1

Updated by okurz about 3 years ago

  • Description updated (diff)
Actions #2

Updated by okurz about 3 years ago

https://github.com/os-autoinst/openQA/pull/4289 might help with retrying but that would need converting obs-rsync to use openqa-cli instead of openqa-client. For this I also created https://github.com/os-autoinst/openqa-trigger-from-obs/pull/140 but that fails tests right now and would need work.

Actions

Also available in: Atom PDF