action #134438: [qe-sap] missing 15-SP2 installation media on https://openqa.suse.de/assets/repo/ - openQA Tests - openSUSE Project Management Tool

Actions

Copy link

action #134438

open

[qe-sap] missing 15-SP2 installation media on https://openqa.suse.de/assets/repo/

Added by emiura 9 months ago. Updated 3 months ago.

Status:

New

Priority:

Normal

Assignee:

Category:

New test

Target version:

Start date:

2023-12-11

Due date:

% Done:

100%

Estimated time:

(Total: 0.00 h)

Difficulty:

Description

Due to refactor on boot_from_pxe code, it was necessary to add MIRROR_HTTP variable on jobgroups. However, SLE 15-SP2 misses either Online media, Full or Installed media under https://openqa.suse.de/assets/repo/ (only s390x and ppc64le could be found).

Subtasks 1 (0 open — 1 closed)

Actions

Copy link

Updated by emiura 9 months ago

Required to fix TEAM-8453

Actions

Copy link

Updated by maritawerner 8 months ago

Project changed from openQA Tests to openQA Infrastructure

Actions

Copy link

Updated by okurz 8 months ago

Project changed from openQA Infrastructure to openQA Tests
Subject changed from missing 15-SP2 installation media on https://openqa.suse.de/assets/repo/ to [qe-sap] missing 15-SP2 installation media on https://openqa.suse.de/assets/repo/
Category set to New test

Anyone with their salt key in https://gitlab.suse.de/openqa/salt-pillars-openqa can add fixed repos in osd:/var/lib/openqa/share/factory/…

Actions

Copy link

Updated by jkohoutek 6 months ago

okurz wrote in #note-3:

Anyone with their salt key in https://gitlab.suse.de/openqa/salt-pillars-openqa can add fixed repos in osd:/var/lib/openqa/share/factory/…

So we does, but day after they are gone, so where it's handled and how?

Actions

Copy link

Updated by okurz 6 months ago

jkohoutek wrote in #note-4:

So we does, but day after they are gone, so where it's handled and how?

You mean directories from osd:/var/lib/openqa/share/factory/repo/fixed are gone? Well, as those are manually handled.

openqa:~ # grep 'fixed' /var/log/audit/audit.log
type=EXECVE msg=audit(1700035201.095:9244129): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-15-Installer-DVD-s390x-GM-DVD1/"
type=EXECVE msg=audit(1700035201.095:9244131): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-15-SP1-Installer-DVD-s390x-GM-DVD1/"
type=EXECVE msg=audit(1700035201.095:9244132): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-12-SP?-SDK-POOL-x86_64-BuildGM-Media1/"
type=EXECVE msg=audit(1700035201.103:9244139): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-12-SP?-SDK-POOL-aarch64-BuildGM-Media1/"
type=EXECVE msg=audit(1700035201.103:9244140): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-15-SP2-Full-s390x-GM-Media1/"
type=EXECVE msg=audit(1700035201.103:9244144): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-12-SP?-SDK-POOL-ppc64le-BuildGM-Media1/"
type=EXECVE msg=audit(1700035201.107:9244145): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-12-SP?-SDK-POOL-s390x-BuildGM-Media1/"
type=EXECVE msg=audit(1700035201.111:9244159): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-15-SP2-Installer-DVD-s390x-GM-DVD1/"
type=EXECVE msg=audit(1700035201.115:9244160): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-12-Server-DVD-s390x-GM-DVD1/"
type=EXECVE msg=audit(1700035201.115:9244166): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-15-SP2-Online-s390x-GM-Media1/"
type=EXECVE msg=audit(1700035201.119:9244168): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-15-SP3-Full-s390x-GM-Media1/"
type=EXECVE msg=audit(1700035201.119:9244169): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-12-SP3-Server-DVD-s390x-GM-DVD1/"
type=EXECVE msg=audit(1700035201.123:9244172): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-15-SP3-Online-s390x-GM-Media1/"
type=EXECVE msg=audit(1700035201.123:9244174): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-12-SP4-Server-DVD-s390x-GM-DVD1/"
type=EXECVE msg=audit(1700035201.123:9244177): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-12-SP5-Server-DVD-s390x-GM-DVD1/"
type=EXECVE msg=audit(1700035201.123:9244178): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-15-SP4-Full-s390x-GM-Media1/"
type=EXECVE msg=audit(1700035201.127:9244181): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-15-SP4-Online-s390x-GM-Media1/"
type=EXECVE msg=audit(1700035201.131:9244182): argc=3 a0="ln" a1="-sf" a2="fixed/SLE-15-SP5-Online-s390x-GM-Media1/"
type=EXECVE msg=audit(1700035280.309:9245194): argc=4 a0="grep" a1="--color=auto" a2="fixed" a3="/var/log/audit/audit.log"

doesn't show any recent deletions at least. Also journalctl -u velociraptor-client --grep fixed --since="2 days ago" did not find anything.

okurz@openqa:~> ls -l /var/lib/openqa/share/factory/repo/fixed/ | grep 'SLE-15-SP2'
drwxr-xr-x 22 geekotest nogroup 4096 Aug 19  2020 SLE-15-SP2-Full-aarch64-Build209.2-Media1
drwxr-xr-x 22 geekotest nogroup 4096 Sep 14  2020 SLE-15-SP2-Full-aarch64-GM-Media1
drwxr-xr-x 23 geekotest nogroup 4096 Aug 19  2020 SLE-15-SP2-Full-ppc64le-Build209.2-Media1
drwxr-xr-x 21 geekotest nogroup 4096 Aug 28  2020 SLE-15-SP2-Full-s390x-Build209.2-Media1
drwxr-xr-x 21 geekotest nogroup 4096 Sep 14  2020 SLE-15-SP2-Full-s390x-GM-Media1
drwxr-xr-x 30 geekotest nogroup 4096 Aug 28  2020 SLE-15-SP2-Full-x86_64-Build209.2-Media1
drwxr-xr-x 30 geekotest nogroup 4096 Sep 14  2020 SLE-15-SP2-Full-x86_64-GM-Media1
dr-xr-xr-x 21 geekotest nogroup 4096 Jun 11  2020 SLE-15-SP2-Installer-DVD-s390x-GM-DVD1
drwxr-xr-x  7 geekotest nogroup 4096 Jun 12  2020 SLE-15-SP2-Online-aarch64-Build209.2-Media1
drwxr-xr-x  7 geekotest nogroup 4096 Sep 14  2020 SLE-15-SP2-Online-aarch64-GM-Media1
drwxr-xr-x  7 geekotest nogroup 4096 Jun 12  2020 SLE-15-SP2-Online-ppc64le-Build209.2-Media1
lrwxrwxrwx  1 root      root      43 Jan  4  2023 SLE-15-SP2-Online-ppc64le-GM-Media1 -> SLE-15-SP2-Online-ppc64le-Build209.2-Media1
drwxr-xr-x  6 geekotest nogroup 4096 Oct 27  2020 SLE-15-SP2-Online-s390x-Build209.2-Media1
drwxr-xr-x  6 geekotest nogroup 4096 Jan 27  2021 SLE-15-SP2-Online-s390x-GM-Media1
dr-xr-xr-x  6 geekotest nogroup 4096 Jun  8  2021 SLE-15-SP2-Online-s390x-QU2-Media1
drwxr-xr-x  7 geekotest nogroup 4096 Jun 12  2020 SLE-15-SP2-Online-x86_64-Build209.2-Media1
drwxr-xr-x  7 geekotest nogroup 4096 Sep 14  2020 SLE-15-SP2-Online-x86_64-GM-Media1

so it seems that e.g. "Installer" is missing for x86_64+aarch64+ppc64le. Do you assume they have been there and are gone just by now?

Actions

Copy link

Updated by jkohoutek 6 months ago

okurz wrote in #note-5:

jkohoutek wrote in #note-4:

So we does, but day after they are gone, so where it's handled and how?

You mean directories from osd:/var/lib/openqa/share/factory/repo/fixed are gone? Well, as those are manually handled.

What I mean is some kind of automation, like RSYNC, which should revert the direct changes on the system.

okurz wrote in #note-5:

doesn't show any recent deletions at least. Also journalctl -u velociraptor-client --grep fixed --since="2 days ago" did not find anything.

Because we place them back already:
SLE-15-SP2-Full-x86_64-GM-Media1/ 2023-11-14 16:01

okurz wrote in #note-5:

so it seems that e.g. "Installer" is missing for x86_64+aarch64+ppc64le. Do you assume they have been there and are gone just by now?

Not sure about aarch64 and ppc64le , but x86_64 was there at 11/13 and gone at 11/14

And beside the 15-SP2's, I just found that there are also missing some 15-SP3's:
No SLE-15-SP3-Online-[ppc64le|x86_64]-GM:

[DIR] SLE-15-SP3-Online-ppc64le-Build187.1-Media1/ 2021-05-10 21:47 - [DIR] SLE-15-SP3-Online-s390x-GM-Media1/ 2021-06-08 09:08 - [DIR] SLE-15-SP3-Online-x86_64-Build176.5-Media1/ 2021-04-19 15:00 - [DIR] SLE-15-SP3-Online-x86_64-Build187.1-Media1/ 2021-05-10 21:47 -

Same for SLE-15-SP3-Full-[ppc64le|x86_64]-GM:

[DIR] SLE-15-SP3-Full-s390x-Build187.1-Media1/ 2021-05-10 21:42 - [DIR] SLE-15-SP3-Full-s390x-GM-Media1/ 2021-05-10 21:42 -

Actions

Copy link

Updated by okurz 6 months ago

Unless you know of a good way how to audit and monitor for any removals I can only think of you asking in #eng-testing if anybody has manually deleted content or knows of automation that could cause this.

Actions

Copy link