QA (public) » openQA Project (public) » openQA Infrastructure (public)

Observation¶

This caused multiple alert e-mails pointing to https://monitor.qa.suse.de/alerting/grafana/d74e764d-6097-4d14-b77c-76c8d1da6ff0/view?orgId=1. The state history can be viewed on https://monitor.qa.suse.de/alerting/list?search=namespace:%22Salt%22%20group:%22inodes%22 and shows that the alert was going on and off 4 times. It was also pending 3 times before that.

Acceptance criteria¶

• AC1: No more alerts about inode utilization

Suggestions¶

• Check whether the alert thresholds are reasonable and possibly adjust them.
• Check whether there were any actual issues with tests that ran around that time (see further details).

Further details¶

Incomplete jobs around that time:

openqa=> select id, result, reason from jobs where t_started >= '2025-02-04 18:00:00' and t_finished <= '2025-02-05 00:00:00' and result in ('incomplete') order by t_started;
    id    |   result   |                                                                                                                                                    reason                                                          

----------+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------
 16655323 | incomplete | died: terminated prematurely, see log output for details
 16655373 | incomplete | api failure: 400 response: Error in tempfile() using template /var/lib/openqa/testresults/16655/16655373-sle-15-SP6-Server-DVD-SAP-Incidents-x86_64-Build:36438:yast2-sap-ha-sles4sap_ensa_supportserver@64bit-4gbr
am/XXXXXXXXXX: Could not create temp file /var/lib/openqa/testresults/16655/16655373-sle-…
 16650790 | incomplete | api failure: 400 response: Error in tempfile() using template /var/lib/openqa/testresults/16650/16650790-sle-15-SP3-GCE-SAP-BYOS-Incidents-saptune-x86_64-Build:37325:rpmlint-sles4sap_gnome_saptune_solutions@gce_
n1_highmem_8/XXXXXXXXXX: Could not create temp file /var/lib/openqa/testresults/16650/166…
 16650795 | incomplete | api failure: 400 response: Error in tempfile() using template /var/lib/openqa/testresults/16650/16650795-sle-15-SP4-Azure-SAP-PAYG-Incidents-x86_64-Build:37325:rpmlint-SAPHanaSR-ScaleUp-PerfOpt-spn@az_Standard_E
4s_v3/XXXXXXXXXX: Could not create temp file /var/lib/openqa/testresults/16650/16650795-s…
 16655545 | incomplete | api failure: 400 response: Error in tempfile() using template /var/lib/openqa/testresults/16655/16655545-sle-15-SP3-BCI-Updates-x86_64-Build2.35_sles15-ltss-image-bci-base_15.3-ltss_on_SLES_15-SP5_podman@64bit/X
XXXXXXXXX: Could not create temp file /var/lib/openqa/testresults/16655/16655545-sle-15-S…
 16655725 | incomplete | api failure: 400 response: Error in tempfile() using template /var/lib/openqa/testresults/16655/16655725-sle-15-SP6-BCI-Updates-x86_64-Build5.3_dotnet-9.0-dotnet-sdk_9.0_on_SLES_15-SP4_docker@64bit/XXXXXXXXXX: C
ould not create temp file /var/lib/openqa/testresults/16655/16655725-sle-15-SP6-BCI-Updat…
 16658339 | incomplete | asset failure: Failed to download SLE-15-SP4-x86_64-Build20250204-1-qam-sle_sap-gnome.qcow2 to /var/lib/openqa/cache/openqa.suse.de/SLE-15-SP4-x86_64-Build20250204-1-qam-sle_sap-gnome.qcow2
 16660457 | incomplete | backend died: QEMU exited unexpectedly, see log for details
 16660458 | incomplete | backend died: QEMU exited unexpectedly, see log for details
 16658857 | incomplete | asset failure: Failed to download sle-15-SP3-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/sle-15-SP3-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2
 16659285 | incomplete | asset failure: Failed to download sle-15-SP4-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/sle-15-SP4-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2
 16659376 | incomplete | asset failure: Failed to download sle-15-SP5-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/sle-15-SP5-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2
 16659503 | incomplete | backend died: QEMU exited unexpectedly, see log for details
 16660024 | incomplete | asset failure: Failed to download autoyast_SLES-15-SP5-x86_64-create_hdd_yast_maintenance_minimal-Build20250204-1-Server-DVD-Updates-64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/autoyast_SLES-15-SP5-x86_6
4-create_hdd_yast_maintenance_minimal-Build20250204-1-Server-DVD-Updates-64bit.qcow2
 16660516 | incomplete | asset failure: Failed to download autoyast_SLES-15-SP5-x86_64-create_hdd_yast_maintenance_minimal-Build20250204-1-Server-DVD-Updates-64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/autoyast_SLES-15-SP5-x86_6
4-create_hdd_yast_maintenance_minimal-Build20250204-1-Server-DVD-Updates-64bit.qcow2
 16659985 | incomplete | backend died: QEMU exited unexpectedly, see log for details
 16660472 | incomplete | asset failure: Failed to download sle-15-SP4-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/sle-15-SP4-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2
 16660482 | incomplete | asset failure: Failed to download sle-15-SP5-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/sle-15-SP5-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2
 16660467 | incomplete | asset failure: Failed to download sle-15-SP3-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/sle-15-SP3-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2
 16660584 | incomplete | asset failure: Failed to download sle-15-SP4-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/sle-15-SP4-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2
 16660585 | incomplete | asset failure: Failed to download sle-15-SP5-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/sle-15-SP5-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2
 16660583 | incomplete | asset failure: Failed to download sle-15-SP3-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/sle-15-SP3-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2
 16660596 | incomplete | asset failure: Failed to download sle-15-SP4-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/sle-15-SP4-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2
 16660597 | incomplete | asset failure: Failed to download sle-15-SP3-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/sle-15-SP3-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2
 16660595 | incomplete | asset failure: Failed to download sle-15-SP5-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/sle-15-SP5-x86_64-min-add-qsec-cc-20250204-1@64bit.qcow2
 16660181 | incomplete | asset failure: Failed to download sle-15-SP6-x86_64-min-add-qsec-20250204-1@64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/sle-15-SP6-x86_64-min-add-qsec-20250204-1@64bit.qcow2
 16660731 | incomplete | asset failure: Failed to download SLES-Packages-16.0-x86_64-Build4.2.iso to /var/lib/openqa/cache/openqa.suse.de/SLES-Packages-16.0-x86_64-Build4.2.iso
 16660905 | incomplete | asset failure: Failed to download SLES-16.0-x86_64-mru-install-minimal-with-addons-Build4.2-agama-installer-64bit.qcow2 to /var/lib/openqa/cache/openqa.suse.de/SLES-16.0-x86_64-mru-install-minimal-with-addons-Bu
ild4.2-agama-installer-64bit.qcow2
(28 rows)