Project

General

Profile

Actions

action #128927

closed

OSD deployment changelog repeatedly mentions old os-autoinst changes, is a worker outdated and unable to install newer packages? size:M

Added by okurz over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
-
Start date:
2023-05-08
Due date:
% Done:

0%

Estimated time:

Description

Observation

Recent example from 2023-05-08:

You can find the details on https://gitlab.suse.de/openqa/osd-deployment/-/jobs/1558014
As always, if you encounter problems, please file tickets on https://progress.opensuse.org.
Have fun, your QA tools team


 * Sun May 07 2023 okurz@suse.com
 - Update to version 4.6.1683294886.8a7e554:
   *


os-autoinst changes:
 * Sun May 07 2023 okurz@suse.com
 - Update to version 4.6.1683277334.0688448:
   * Enable VNC endian conversion when worker is big-endian
   * Add support for serial device of s390x in testapi
   * dist: Fix unresolved dependency on chattr in OBS
   * Show initialization errors of `isotovideo` again
   * Test sending forced VNC update request explicitly
   * container: Distinguish jq and non-jq variants in BuildTag
   * Drop setting of TESSDATA_PREFIX in invoke-tests
   * Replace Perl::Critic::{Freenode,Community}
   * Add workaround to prevent `t/29-backend-generalhw.t` being unstable
   * Cover all code of `console.pm`
   * Fix CMake warning about ordering
   * ci: Improve marking uncoverable statements
   * Drop Leap-15.3 from OBS CI

I have seen the oldest os-autoinst changes mentioned here in before, e.g. "Drop Leap-15.3 from OBS CI". I assume we have the same problem that we had already in the past where at least one worker was unable to update packages due to ppc64le builds failing on devel:openQA

Acceptance criteria

  • AC1: The problematic worker instance is receiving updates again with the same frequency as other workers

Suggestions

  • Look into devel:openQA for build failures and fix them
  • DONE Find the old ticket about "changelog" -> #124658
  • Look into the deployment pipeline details
  • Check that all salt machines are up-to-date or fix the found problem(s)

Related issues 2 (0 open2 closed)

Related to openQA Infrastructure (public) - action #120372: OSD deployment fails presumably when handling the changelogResolvedokurz2022-11-14

Actions
Related to openQA Infrastructure (public) - action #124658: [osd-deployment] missing os-autoinst changes in deployment and changelog emails , build failure on ppc64le in OBS size:MResolvedmkittler2023-02-16

Actions
Actions #1

Updated by okurz over 1 year ago

  • Related to action #120372: OSD deployment fails presumably when handling the changelog added
Actions #2

Updated by okurz over 1 year ago

  • Related to action #124658: [osd-deployment] missing os-autoinst changes in deployment and changelog emails , build failure on ppc64le in OBS size:M added
Actions #3

Updated by okurz over 1 year ago

  • Description updated (diff)
Actions #4

Updated by mkittler over 1 year ago

  • Subject changed from OSD deployment changelog repeatedly mentions old os-autoinst changes, is a worker outdated and unable to install newer packages? to OSD deployment changelog repeatedly mentions old os-autoinst changes, is a worker outdated and unable to install newer packages? size:M
  • Description updated (diff)
  • Status changed from New to Workable
Actions #5

Updated by mkittler over 1 year ago

  • Assignee set to mkittler
Actions #6

Updated by mkittler over 1 year ago

The worker that was used to compute the changelog was QA-Power8-4-kvm. The worker looks generally good, e.g. zypper dup --allow-vendor-change worked without any conflicts to resolve. I also don't see any PowerPC build failures at the moment.

salt -C 'G@roles:worker' cmd.run 'rpm -q os-autoinst' shows that all workers are on the same os-autoinst version. The same counts for the openQA-worker packages except that they are now a little bit ahead on the PowerPC workers as I've invoked zypper dup there for testing purposes.

The worker openqaworker1.qe.nue2.suse.org has also a newer openQA-worker package. I'll have a look why that is.


Maybe we should always pick an x86_64 worker to generate the changelog for better consistency?

Actions #7

Updated by mkittler over 1 year ago

On openqaworker1.qe.nue2.suse.org there was still openqa-auto-update.timer and openqa-continuous-update.timer running (in contrast to other OSD workers). For consistency, I disabled/stopped those services.

Actions #8

Updated by mkittler over 1 year ago

  • Status changed from Workable to Feedback

Maybe we should always pick an x86_64 worker to generate the changelog for better consistenc

I think it would be beneficial so I went ahead and created https://gitlab.suse.de/openqa/osd-deployment/-/merge_requests/53.

Actions #9

Updated by okurz over 1 year ago

merged

Actions #10

Updated by mkittler over 1 year ago

  • Status changed from Feedback to Resolved

Some time has passed so I ran the salt commands mentioned in my comment from last week again. Now everything is on the same version. The MR has been merged as well. So I suppose the issue can be considered resolved.

Actions

Also available in: Atom PDF