QA » openQA Project » openQA Infrastructure

Both s390zl12+13 will have double the original memory amount. Created https://gitlab.suse.de/openqa/salt-pillars-openqa/-/merge_requests/761 for re-enabling the previously disabled instances as part of #153958.

After that waiting for https://gitlab.suse.de/OPS-Service/salt/-/merge_requests/4951 to have more s390x kvm instances DHCP/DNS entries prepared, then enable more instances in workerconf.sls

s390zl12 and s390zl13 have been upgraded and now have 160GB RAM each (double than before) and 6.0 Processors (previously 4.0)

I have prepared and reserved 20 more instances for future uses with https://gitlab.suse.de/OPS-Service/salt/-/merge_requests/4951

https://suse.slack.com/archives/C02CANHLANP/p1712228880986239

(Oliver Kurz) @Matthias Griessmeier will you follow-up with https://gitlab.suse.de/OPS-Service/salt/-/merge_requests/4951 regarding DHCP pool adjustement or do we need to take over?

Doesn't go well. https://openqa.suse.de/admin/workers/3087 shows no successful jobs on the new instances. In particular https://openqa.suse.de/tests/14103625#step/bootloader_zkvm/44 states

# Test died: Error connecting to VNC server <s390kvm115.oqa.prg2.suse.org:5901>: IO::Socket::INET: connect: No route to host at /usr/lib/os-autoinst/testapi.pm line 1690.

I checked all the s390-kvm workers and saw consistent failures on s390kvm100…s390kvm119 but I have also seen s390kvm093 consistently failing https://openqa.suse.de/admin/workers/2650, not sure about that one.

I created https://gitlab.suse.de/openqa/salt-pillars-openqa/-/merge_requests/784 for mitigation, will merge and restart all related failures with

for i in WORKER="worker33 worker35 worker40"; do host=openqa.suse.de failed_since=2024-04-19 result="result='failed'" comment="label:poo158170" ./openqa-advanced-retrigger-jobs; done

Feel welcome to block on https://jira.suse.com/browse/ENGINFRA-4030 "suttner1.oqa.prg2.suse.org+suttner2.oqa.prg2.suse.org times are both out of sync with NTP causing DHCP-failover to fail" any time and escalate to a line manager of your choice :)