Project

General

Profile

action #102062

check state of packages fails with test '' = clean

Added by cdywan 3 months ago. Updated 2 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
Start date:
2021-11-08
Due date:
2021-12-04
% Done:

0%

Estimated time:

Description

Observation

This morning check state of packages failed:

.++ for i in {1..7}
++++ uname -i
2423+++ curl -s 'https://api.opensuse.org/public/build/devel:openQA/_result?repository=openSUSE_Leap_15.2&arch=x86_64'
[...]
++ echo 'Retry 7 after sleep ...'
2517Retry 7 after sleep ...
2518++ sleep 300
[...]
++ echo '$ test "$status" = "clean"'
2819$ test "$status" = "clean"
2820++ test '' = clean
2821+++ kill %1

Acceptance criteria

  • AC1: Package check concludes with a meaningful result
  • AC2: Package check succeeds

Suggestion

History

#1 Updated by mkittler 3 months ago

Judging by the log it is the same problem as https://github.com/os-autoinst/openQA/pull/4351#issuecomment-962900388. So this is already fixed and retrying the job helped.

Not sure whether it is worth improving the output. One had to scroll a little bit to the top but the source of the problem was actually apparent.

#2 Updated by cdywan 3 months ago

mkittler wrote:

Not sure whether it is worth improving the output. One had to scroll a little bit to the top but the source of the problem was actually apparent.

I suspect it was not apparent to me. Can you elaborate what one should have seen here?

#3 Updated by okurz 3 months ago

  • Target version set to Ready

#4 Updated by mkittler 3 months ago

  • Status changed from New to Feedback
  • Assignee set to mkittler
  • Target version deleted (Ready)

I saw the same dependency resolution error as okurz mentioned in the PR:

    <status package="openQA" code="unresolvable">
      <details>have choice for libpq.so.5()(64bit) needed by perl-DBD-Pg: libpq5 postgresql14-devel-mini, have choice for libpq.so.5()(64bit) needed by postgresql12-server: libpq5 postgresql14-devel-mini, have choice for libpq5 &gt;= 12.6 needed by postgresql12-server: libpq5 postgresql14-devel-mini, have choice for libpq.so.5()(64bit) needed by postgresql12: libpq5 postgresql14-devel-mini, have choice for libpq5 &gt;= 12.6 needed by postgresql12: libpq5 postgresql14-devel-mini</details>
    </status>

And the following:

++ grep 'status package'
++ grep -v '\(disabled\|succeeded\|excluded\)'
    <status package="openQA" code="unresolvable">
    <status package="openQA:openQA-test" code="unresolvable">

Which makes it clear that these two packages were unresolvable (and they should be succeeded or at least disabled/excluded).

#5 Updated by okurz 3 months ago

  • Target version set to Ready

#6 Updated by tinita 3 months ago

Regarding the scrolling up:

The dots come from this line (see the link above the line for what this is needed):
https://gitlab.suse.de/openqa/osd-deployment/-/blob/master/.gitlab-ci.yml#L49

.gitlab_workaround: &gitlab_workaround |
  (
  trap "kill %1" EXIT; while true; do sleep 1 && echo -n .; done &
  set -ex

(note that the -n for the echo did have an effect a while ago, and the dots were all on the same line, but someday gitlab added a linebreak itself)

I suggested that we increase the sleep time from 1 to 5 seconds. Since that is running in the background it won't affect the actual pipeline itself, it might just take 4 more seconds at the end (the pipeline in question took over 35 minutes).
https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4119#note_257314431 suggests that a higher sleep time will still be good enough to workaround the bug.

That would considerably reduce the number of lines with a dot. I find it annoying how much I have to scroll to find actual error messages.

Alternatively we could check in which version(s) of gitlab and/or related software the bug was fixed, and which versions we have running. Then we could eventually remove the workaround completely.

#7 Updated by cdywan 3 months ago

  • Description updated (diff)
  • Assignee changed from mkittler to tinita

Discussed this briefly with marius/oli/tina/sebastian and as an improvement, the script could show a URL to the OBS web UI which would provide the context for what the script is also checking (maybe mapped from the API URL).

#8 Updated by tinita 2 months ago

  • Status changed from Feedback to In Progress

#9 Updated by openqa_review 2 months ago

  • Due date set to 2021-12-04

Setting due date based on mean cycle time of SUSE QE Tools

#10 Updated by tinita 2 months ago

  • Status changed from In Progress to Feedback

#11 Updated by tinita 2 months ago

MR was approved and I just merged it

#12 Updated by tinita 2 months ago

  • Status changed from Feedback to Resolved

Pipeline succeeded

Also available in: Atom PDF