action #109851
closedos-autoinst was removed from o3 openqaworker7
Description
Updated by okurz over 2 years ago
- Assignee set to mkittler
mkittler forced installation of os-autoinst. zypper said it would be still installed but /usr/bin/isotovideo was gone.
I retriggered incomplete jobs with worker=openqaworker7 openqa-advanced-retrigger-jobs. https://openqa.opensuse.org/tests/overview?arch=&flavor=&machine=&test=&modules=&module_re=&distri=microos&distri=opensuse&version=Tumbleweed&build=20220411&groupid=1# looks good now.
Updated by okurz over 2 years ago
First report of the problem in https://matrix.to/#/!ilXMcHXPOjTZeauZcg:libera.chat/$FZWJrLsuo2zwoPL3nHQha60RZh3on-wzW3x5DKB0MBU
/var/log/zypp/history says:
2022-04-12 03:47:22|remove |os-autoinst|4.6.1649692679.6d936fdc-lp153.1200.1|x86_64||
fvogt suggests to use /etc/zypp/systemCheck.d/.
Related story is #105379 by Martchus
DimStar: zypper in -r limits to that repo, deps across repos are benig ignored; zypper in --from takes package preferred from the repo, but allows to step out to others if needed
fvogt: it should probably just use zypper dup
Martchus: DimStar: ah, good to know - so just updating from a repo at least via-ris a bad idea indeed
Martchus: fvogt: yes, I'll switch it to just use zypper dup
Martchus: or betterzypper -n dup --replacefiles --auto-agree-with-licenses --force-resolution --download-in-advancein consistency with openqa-auto-update
fvogt: Please no
fvogt: --force-resolution and --replacefiles are evil
fvogt: That's just asking for trouble
Martchus: then I'll omit them
fvogt: I think it's really sigpipe which broke it
fvogt: That might've caused zypper to cache empty/incomplete repo data
fvogt: 2022-04-12 03:47:02 shows that sigpipe broke zypper during refresh
fvogt: I see the issue
fvogt: | grep -q 'is up to date' is indeed broken
fvogt: grep exits after that and zypper dies
fvogt: Could still be a bug in zypper, but should be fixed in the script as well
Martchus: likely it is better to useout=$(zypper -n ref -r "$repo_name")then
fvogt: echo "requires:openQA-worker" > /etc/zypp/systemCheck.d/openqa.check
fvogt: Now removing os-autoinst results in
fvogt: Problem: This request will break your system!
fvogt: Only works without --force-resolution
Updated by mkittler over 2 years ago
- Related to action #105379: Continuous deployment of o3 workers - one worker first size:M added
Updated by mkittler over 2 years ago
- Status changed from In Progress to Feedback
The timer has been disabled again. Since it is nearing the end of the day I'll refrain from enabling it again and will instead try the new version tomorrow. It'll hopefully work better. I'll keep the ticket in feedback until I'm sure about it and will monitor the log of openqaworker7 more closely.
Updated by mkittler over 2 years ago
The PR https://github.com/os-autoinst/openQA/pull/4602 has now been enabled on openqaworker7. If something goes wrong, just have a look at the openqa-continuous-update.service log or the zyppper log and stop/disable openqa-continuous-update.timer. I'm monitoring what's going on (as root user within a tmux session). So far it looks good, it has already updated os-autoinst-distri-opensuse-deps.
Updated by mkittler over 2 years ago
- Status changed from Feedback to Resolved
The worker looks still good. Since the urgent issue has been resolved I leave further handling to #105379.