QA » openQA Project » openQA Tests

Observation¶

Various openQA tests fail on s390x-kvm, e.g. in scenario sle-15-SP4-Server-DVD-Updates-s390x-mau-extratests-phub@s390x-kvm fails in
python_scientific
with

command 'zypper -n in python3 python3-numpy python3-scipy | cat; ( exit ${PIPESTATUS[0]} )' timed out

https://openqa.suse.de/tests/13047998#step/kubectl/14 is particularly interesting because already when trying to refresh repos this goes into a timeout

Reproducible¶

Fails since Build 20231207-1
2023-12-08

Expected result¶

Last good: 20231206-1 (or more recent)

Problem¶

https://openqa.suse.de/tests/12996343#investigation shows

diff_to_last_good   

…
-   "BUILD" : "20231206-1",
+   "BUILD" : "20231207-1",
-   "HDD_1" : "SLES-15-SP4-s390x-mru-install-minimal-with-addons-Build20231206-1-Server-DVD-Updates-s390x-kvm.qcow2",
+   "HDD_1" : "SLES-15-SP4-s390x-mru-install-minimal-with-addons-Build20231207-1-Server-DVD-Updates-s390x-kvm.qcow2",
+   "LEGACY_TEST_ISSUES" : "31724",
+   "LEGACY_TEST_REPOS" : "http://download.suse.de/ibs/SUSE:/Maintenance:/31724/SUSE_Updates_SLE-Module-Legacy_15-SP4_s390x/",
…
-   "NEEDLES_GIT_HASH" : "52c3ca3c09d640ded913f02d4c71178552c5bc07",
+   "NEEDLES_GIT_HASH" : "78d2d6be3d0d587e9e5f0cc1f3edc7cbd312d523",
…
-   "REPOHASH" : "24e73cca5581cc19eedea1937953f3d0",
+   "REPOHASH" : "bef082847e1ac3bde609f8f317c8b877",
…
-   "SUT_IP" : "s390kvm090.oqa.prg2.suse.org",
+   "SUT_IP" : "s390kvm080.oqa.prg2.suse.org",
-   "TEST_GIT_HASH" : "2139f98b566ccda1c6da21fa079f6e074f8c6b7c",
+   "TEST_GIT_HASH" : "a76c65d6134fd10718e5285588d998e10964c46c",
-   "UEFI_PFLASH_VARS" : "SLES-15-SP4-s390x-mru-install-minimal-with-addons-Build20231206-1-Server-DVD-Updates-s390x-kvm-uefi-vars.qcow2",
+   "UEFI_PFLASH_VARS" : "SLES-15-SP4-s390x-mru-install-minimal-with-addons-Build20231207-1-Server-DVD-Updates-s390x-kvm-uefi-vars.qcow2",
-   "VIRSH_GUEST" : "10.145.10.90",
-   "VIRSH_HOSTNAME" : "s390zl13.oqa.prg2.suse.org",
+   "VIRSH_GUEST" : "10.145.10.80",
+   "VIRSH_HOSTNAME" : "s390zl12.oqa.prg2.suse.org",
-   "VIRSH_MAC" : "52:54:00:9d:1a:57",
+   "VIRSH_MAC" : "52:54:00:77:af:b4",
-   "WORKER_CLASS" : "s390-kvm,s390-kvm-sle12-mm,s390zl13,s390kvm090,prg,prg2,worker33,cpu-x86_64,cpu-x86_64-v2,cpu-x86_64-v3",
-   "WORKER_HOSTNAME" : "worker33.oqa.prg2.suse.org",
-   "WORKER_ID" : 2647,
+   "WORKER_CLASS" : "s390-kvm,s390-kvm-sle12-mm,s390zl12,s390kvm080,prg,prg2,worker31,cpu-x86_64,cpu-x86_64-v2,cpu-x86_64-v3",
+   "WORKER_HOSTNAME" : "worker31.oqa.prg2.suse.org",
+   "WORKER_ID" : 2567,

last_good   12986251
needles_diff_stat   

 autoyast-system-login-console-minimal-20231017.json |  15 +++++++++++++++
 autoyast-system-login-console-minimal-20231017.png  | Bin 0 -> 71874 bytes
 2 files changed, 15 insertions(+)

needles_log 

+ aa361be42 Adding a new needle for poo 133997.

test_diff_stat  

 data/publiccloud/terraform/azure.tf           | 43 +++---------------
 data/publiccloud/terraform/azure_nfstest.tf   | 44 +++----------------
 data/publiccloud/terraform/gce.tf             |  7 +--
 lib/publiccloud/provider.pm                   |  2 +
 lib/qesapdeployment.pm                        | 63 +++++++++++++++------------
 schedule/security/alp/container_selinux.yaml  |  5 +++
 schedule/security/alp/fde_misc.yaml           |  6 +++
 schedule/security/alp/fips_crypt_core.yaml    | 54 +++++++++++++++++++++++
 schedule/security/fips_crypt_core.yaml        | 46 ++++---------------
 schedule/security/selinux.yaml                |  5 +++

Show more
test_log    

+ baa7c2038 Remove subscription_id from az related qesap API
+ d1ce35810 openldap_configuration: only disbale nscd if installed
+ fd8310c6a Update BCI openQA test runs to new version scheme
+ 711fd1271 gdb: disable debuginfod url during test
+ 54ad70f0c Refactor fips/fips_setup.pm

+ ca6eb7aef Use suseconnect_scc on SLEM
+ 1cbd94785 Public Cloud: Reuse resources in Google and Azure

so there are some changes in os-autoinst-distri-opensuse, some schedule changes and in the diff of settings I see that "LEGACY_TEST_REPOS" now appeared in the "first bad" which was not there in before, maybe related?

Impact¶

It seems many if not all s390x-kvm SLE maintenance tests are impacted

Further details¶

Always latest result in this scenario: latest