action #112595
closedcontinous deployment installed old version of openQA due to timeout accessing a repo size:M
Description
Observation¶
On o3 from journalctl --since=2022-06-16 -u openqa-continuous-update
Jun 16 18:34:01 ariel systemd[1]: Started Continuously deploys openQA, see https://progress.opensuse.org/issues/105379.
Jun 16 18:34:02 ariel openqa-continuous-update[27868]: devel:openQA looks good for Leap 15.3 (x86_64)
Jun 16 18:35:02 ariel openqa-continuous-update[27880]: Problem retrieving files from 'devel:openQA:Leap'.
Jun 16 18:35:02 ariel openqa-continuous-update[27880]: Timeout exceeded when accessing 'http://download.opensuse.org/repositories/devel:/openQA:/Leap:/15.3/openSUSE_Leap_15.3/repodata/repomd.xml'.
Jun 16 18:35:02 ariel openqa-continuous-update[27880]: Please see the above error message for a hint.
Jun 16 18:35:02 ariel openqa-continuous-update[27880]: Warning: Skipping repository 'devel:openQA:Leap' because of the above error.
Jun 16 18:35:09 ariel openqa-continuous-update[27880]: Some of the repositories have not been refreshed because of an error.
Jun 16 18:35:09 ariel openqa-continuous-update[27880]: Loading repository data...
Jun 16 18:35:11 ariel openqa-continuous-update[27880]: Reading installed packages...
Jun 16 18:35:16 ariel openqa-continuous-update[27880]: Warning: You are about to do a distribution upgrade with all enabled repositories. Make sure these repositories are compatible before you continue. See 'man zypper' for more information about this command.
Jun 16 18:35:16 ariel openqa-continuous-update[27880]: Computing distribution upgrade...
Jun 16 18:35:17 ariel openqa-continuous-update[27880]: The following 18 packages are going to be downgraded:
Jun 16 18:35:17 ariel openqa-continuous-update[27880]: openQA openQA-client openQA-common perl-Carp-Always perl-Compress-Raw-Lzma perl-Cpanel-JSON-XS perl-DBD-Pg perl-DBI perl-DBIx-Class-DeploymentHandler perl-Minion perl-Mojolicious perl-Mojolicious-Plugin-Asset>
Jun 16 18:35:17 ariel openqa-continuous-update[27880]: The following 18 packages are going to change vendor:
Jun 16 18:35:17 ariel openqa-continuous-update[27880]: openQA
Jun 16 18:35:17 ariel openqa-continuous-update[27880]: obs://build.opensuse.org/devel:openQA -> openSUSE
so an old version of openQA was installed. Fixed by running zypper dup shortly after.
Acceptance criteria¶
- AC1: Failures to refresh one repo lead to aborting the upgrade attempt
Suggestions¶
- Likely the problem happens due to an implicit refresh in https://github.com/os-autoinst/openQA/blob/master/script/openqa-continuous-update#L33 of other repos to fail which surprisingly did not seem to abort the
zypper dupprocess. Try to replicate this error condition and either (report a bug to) fix the zypper behaviour or work around it. - Report an upstream issue or comment on an existing one
Updated by okurz over 2 years ago
Network issues should always be expected by zypper guys state that they won't retry on 404 or similar which is also understandable :) I expect that ref && dup -no-refresh would have behaved correctly here. Because here it seems it was the implicit refresh in the zypper dup call that failed on one repo and zypper dup continued regardless. So the consequence would be to always only use zypper ref && zypper --no-refresh <any_other_operation> but that means that the default functionality of zypper would be wrong and broken?
Updated by okurz over 2 years ago
- Related to action #112232: [tools] Multiple recurring failures due to zypper failing to download packages temporarily added
Updated by livdywan over 2 years ago
- Subject changed from continous deployment installed old version of openQA due to timeout accessing a repo to continous deployment installed old version of openQA due to timeout accessing a repo size:M
- Description updated (diff)
- Status changed from New to Workable
Updated by mkittler over 2 years ago
- Status changed from Workable to In Progress
Updated by mkittler over 2 years ago
I've also created an upstream issue: https://github.com/openSUSE/zypper/issues/446
Updated by openqa_review over 2 years ago
- Due date set to 2022-07-09
Setting due date based on mean cycle time of SUSE QE Tools
Updated by mkittler over 2 years ago
- Status changed from In Progress to Feedback
The PR https://github.com/os-autoinst/openQA/pull/4722 has been deployed on o3. I'll close this issue in a few days unless there are any regressions. I wouldn't wait for the upstream issue to be resolved, though.
Updated by mkittler over 2 years ago
- Status changed from Feedback to Resolved
o3 survived last week and is up to date. Logs of openqa-continuous-update also look as usual.