Project

General

Profile

action #90062

osd deployment fails in "record package versions pre-deploy" maybe because a kernel update is pending?

Added by okurz 4 months ago. Updated 3 months ago.

Status:
Workable
Priority:
Low
Assignee:
-
Target version:
Start date:
2021-03-15
Due date:
% Done:

0%

Estimated time:

Description

Observation

https://gitlab.suse.de/openqa/osd-deployment/-/jobs/361054 fails but everything looks ok except maybe

        Note: System reboot required.

maybe we can ignore this status for exit code evaluation as the deployment should not be concerned about reboot-needing updates as the automatic reboots that are triggered weekly should handle that.

Acceptance criteria

  • AC1: The pipeline step only fails if actions by reviewers are necessary

Suggestions

  • Try to reproduce
  • Research exit code for zypper based on reboot needing status
  • Ignore specific exit status of "needs reboot" for this pipeline

History

#1 Updated by mkittler 4 months ago

Judging by zypper's code I double it returns a non-zero exit code just because a system reboot is required.

The log of the problematic zypper call:

.+ sudo zypper -n dup --download-only --details --force-resolution
.Loading repository data...
Reading installed packages...
..Warning: You are about to do a distribution upgrade with all enabled repositories. Make sure these repositories are compatible before you continue. See 'man zypper' for more information about this command.
Computing distribution upgrade...
…
    6 packages to upgrade, 3 new.
    Overall download size: 0 B. Already cached: 66.1 MiB. Download only.

        Note: System reboot required.
    Continue? [y/n/v/...? shows all options] (y): y
    In cache os-autoinst-4.6.1615462203.793fdffe-lp152.687.1.aarch64.rpm (1/9), 281.9 KiB (979.0 KiB unpacked)
    In cache os-autoinst-openvswitch-4.6.1615462203.793fdffe-lp152.687.1.aarch64.rpm (2/9),  82.5 KiB (  9.4 KiB unpacked)
    In cache os-autoinst-distri-opensuse-deps-1.1615779816.193cd8c31-lp152.7160.1.noarch.rpm (3/9),   8.1 KiB (    0   B unpacked)
    In cache openQA-common-4.6.1615570408.acacd6dbc-lp152.3772.1.noarch.rpm (4/9), 359.2 KiB (376.1 KiB unpacked)
    In cache openQA-client-4.6.1615570408.acacd6dbc-lp152.3772.1.noarch.rpm (5/9), 285.9 KiB ( 57.6 KiB unpacked)
    In cache openQA-worker-4.6.1615570408.acacd6dbc-lp152.3772.1.noarch.rpm (6/9), 316.9 KiB (152.5 KiB unpacked)
    In cache kernel-default-5.3.18-lp152.66.2.aarch64.rpm (7/9),  64.7 MiB (330.8 MiB unpacked)
    In cache dpdk-kmp-default-19.11.4_k5.3.18_lp152.41-lp152.2.8.1.aarch64.rpm (8/9),  86.7 KiB ( 85.4 KiB unpacked)
    In cache crash-kmp-default-7.2.8_k5.3.18_lp152.36-lp152.3.2.1.aarch64.rpm (9/9),  48.1 KiB ( 11.7 KiB unpacked)
.+++ kill %1
section_end:1615787670:step_script
section_start:1615787670:cleanup_file_variables
Cleaning up file based variables
section_end:1615787670:cleanup_file_variables
ERROR: Job failed: command terminated with exit code 1


A good run looks like this and the restart note is present as well:

+ sudo zypper -n dup --download-only --details --force-resolution
..Loading repository data...
Reading installed packages...
..Warning: You are about to do a distribution upgrade with all enabled repositories. Make sure these repositories are compatible before you continue. See 'man zypper' for more information about this command.
Computing distribution upgrade...
…
    6 packages to upgrade, 3 new.
    Overall download size: 66.1 MiB. Already cached: 0 B. Download only.

        Note: System reboot required.
    Continue? [y/n/v/...? shows all options] (y): y
    Retrieving package os-autoinst-4.6.1616752214.24ec8f98-lp152.691.1.aarch64 (1/9), 282.5 KiB (982.0 KiB unpacked)
    Retrieving: os-autoinst-4.6.1616752214.24ec8f98-lp152.691.1.aarch64.rpm [done]
    Retrieving package os-autoinst-openvswitch-4.6.1616752214.24ec8f98-lp152.691.1.aarch64 (2/9),  82.1 KiB (  9.4 KiB unpacked)
    Retrieving: os-autoinst-openvswitch-4.6.1616752214.24ec8f98-lp152.691.1.aarch64.rpm [done]
    Retrieving package os-autoinst-distri-opensuse-deps-1.1617013784.5e6649c41-lp152.7233.1.noarch (3/9),   8.1 KiB (    0   B unpacked)
    Retrieving: os-autoinst-distri-opensuse-deps-1.1617013784.5e6649c41-lp152.7233.1.noarch.rpm [done]
    Retrieving package openQA-common-4.6.1617015091.630afab39-lp152.3811.1.noarch (4/9), 361.3 KiB (377.3 KiB unpacked)
    Retrieving: openQA-common-4.6.1617015091.630afab39-lp152.3811.1.noarch.rpm [done]
    Retrieving package openQA-client-4.6.1617015091.630afab39-lp152.3811.1.noarch (5/9), 287.7 KiB ( 57.6 KiB unpacked)
    Retrieving: openQA-client-4.6.1617015091.630afab39-lp152.3811.1.noarch.rpm [done]
    Retrieving package openQA-worker-4.6.1617015091.630afab39-lp152.3811.1.noarch (6/9), 318.2 KiB (154.8 KiB unpacked)
    Retrieving: openQA-worker-4.6.1617015091.630afab39-lp152.3811.1.noarch.rpm [.done (1.2 KiB/s)]
    Retrieving package kernel-default-5.3.18-lp152.66.2.aarch64 (7/9),  64.7 MiB (330.8 MiB unpacked)
    Retrieving: kernel-default-5.3.18-lp152.66.2.aarch64.rpm [.done (37.9 MiB/s)]
    Retrieving package dpdk-kmp-default-19.11.4_k5.3.18_lp152.41-lp152.2.8.1.aarch64 (8/9),  86.7 KiB ( 85.4 KiB unpacked)
    Retrieving: dpdk-kmp-default-19.11.4_k5.3.18_lp152.41-lp152.2.8.1.aarch64.rpm [done]
    Retrieving package crash-kmp-default-7.2.8_k5.3.18_lp152.36-lp152.3.2.1.aarch64 (9/9),  48.1 KiB ( 11.7 KiB unpacked)
    Retrieving: crash-kmp-default-7.2.8_k5.3.18_lp152.36-lp152.3.2.1.aarch64.rpm [done]
++ echo '$ )'
+++ kill %1
$ )
Cleaning up file based variables 00:00
Job succeeded

I suspect the following log lines show the actual error:

.+ sudo salt -C G@roles:worker cmd.run 'rpm -qa | grep "\(os-autoinst\|openQA-worker\)" > /tmp/osd-deployment_rpm_q && zypper -n dup --download-only --details --force-resolution'
.....openqaworker3.suse.de:
    System management is locked by the application with pid 1286 (zypper).
    Close this application before trying again.

Maybe this ticket is the other side of #90170?

#2 Updated by okurz 3 months ago

  • Priority changed from Normal to Low

#3 Updated by okurz 3 months ago

  • Target version changed from Ready to future

Also available in: Atom PDF