Project

General

Profile

action #76768

HDD images are cleaned-up too early for aarch64 Tumbleweed

Added by ggardet_arm 9 months ago. Updated 9 months ago.

Status:
Resolved
Priority:
High
Assignee:
Target version:
Start date:
2020-10-29
Due date:
% Done:

0%

Estimated time:

Description

HDD images are cleaned-up too early for aarch64 Tumbleweed.
For example in qemu test, hdd image from create_hdd_textmode is not available anymore https://openqa.opensuse.org/tests/1453577


Related issues

Related to openQA Infrastructure - action #65019: ppc64 debuginfo repo is not stripped for TumbleweedResolved2020-03-30

History

#1 Updated by mkittler 9 months ago

Just for the record, it seems that the create_hdd_textmode job really created and uploaded the image (opensuse-Tumbleweed-aarch64-20201028-textmode@aarch64.qcow2: Processing chunk 900/900, avg. speed ~406.312 KiB/s). So an issue here can be outruled.

I doubt that this is an aarch64 specific issue. Our cleanup code considers the architectures equally. Of course the size limit of the "openSUSE Tumbleweed AArch64" job group might not be big enough. However, the cleanup algorithm actually shouldn't consider assets of pending jobs (state is not "done" or "cancelled") at all anyways. I've also double checked some time ago whether that's actually the case when a similar issue came up. Nevertheless the asset has been removed by the cleanup algorithm:

martchus@ariel:~> xzgrep opensuse-Tumbleweed-aarch64-20201028-textmode@aarch64.qcow2 /var/log/openqa_gru.2.xz
[2020-10-29T03:00:12.0988 UTC] [debug] [pid:28083] Checking whether asset hdd/opensuse-Tumbleweed-aarch64-20201028-textmode@aarch64.qcow2 (899416064) fits into group 3 (418768782)
[2020-10-29T03:00:16.0400 UTC] [info] [pid:28083] Removing asset hdd/opensuse-Tumbleweed-aarch64-20201028-textmode@aarch64.qcow2 (belonging to job groups: 3)
[2020-10-29T03:00:16.0421 UTC] [info] [pid:28083] GRU: removed /var/lib/openqa/share/factory/hdd/opensuse-Tumbleweed-aarch64-20201028-textmode@aarch64.qcow2

(Job group 3 is "openSUSE Tumbleweed AArch64".)

So there's either a bug in the cleanup algorithm after all or the asset hasn't been correctly associated with the pending jobs.

#2 Updated by okurz 9 months ago

  • Related to action #65019: ppc64 debuginfo repo is not stripped for Tumbleweed added

#3 Updated by okurz 9 months ago

  • Status changed from New to Feedback
  • Assignee set to ggardet_arm
  • Target version set to future

isn't that a duplicate of #71827 ?

but
ggardet_arm the job group for Tumbleweed aarch64 already has a quota of 200GB which is fully occupied and main contributors are the wasteful copies of e.g. debuginfo packages, see #65019 :

repo/openSUSE-Tumbleweed-oss-armv7hl-Snapshot20201028-debuginfo 33.77 GiB

as you are already working on that and https://github.com/os-autoinst/openqa-trigger-from-obs/pull/102 was merged in the good case you already fixed it for any subsequent build :) I suggest you monitor the situation accordingly.

I am good at assigning tickets back to you, right? ;)

#4 Updated by ggardet_arm 9 months ago

  • Status changed from Feedback to Resolved

The PR did not fix the other ticket, but the problem in this ticket vanished.

Also available in: Atom PDF