QA » openQA Project

Observation¶

Because of occasional failures due to full cache Reason: cache failure: Cache service queue already full (5).
I tried to use CACHE_MIN_FREE_PERCENTAGE based on http://open.qa/docs/#asset-caching to have always some free cache.
Added this variables in workers.ini.

CACHELIMIT=120
CACHE_MIN_FREE_PERCENTAGE=20

Steps to reproduce¶

Define CACHELIMIT & CACHE_MIN_FREE_PERCENTAGE in workers.ini and restart cache services and workers.

Problem¶

Defining CACHE_MIN_FREE_PERCENTAGE broke the cache service.
Cache is empty but the service tries to purge assets, following download is successful but the test fails.

[2022-08-17T23:49:18.864398+02:00] [info] +++ setup notes +++
[2022-08-17T23:49:18.864886+02:00] [info] Running on openqaworker15:22 (Linux 5.3.18-150300.59.87-default #1 SMP Thu Jul 21 14:31:28 UTC 2022 (cc90276) x86_64)
[2022-08-17T23:49:18.874560+02:00] [debug] Found ISO, caching SLE-12-SP4-SAP-DVD-x86_64-GM-DVD1.iso
[2022-08-17T23:49:18.880314+02:00] [info] Downloading SLE-12-SP4-SAP-DVD-x86_64-GM-DVD1.iso, request #17 sent to Cache Service
[2022-08-17T23:49:29.056759+02:00] [info] Download of SLE-12-SP4-SAP-DVD-x86_64-GM-DVD1.iso processed:
[info] [#17] Cache size 0 Byte + needed 0 Byte exceeds limit of 120 GiB, purging least used assets
[info] [#17] Purging "/var/lib/openqa/cache/openqaworker15.qa.suse.cz/SLE-15-SP5-Full-x86_64-Build12.1-Media1.iso.sha256" because we need space for new assets, reclaiming 110 Byte
[info] [#17] Cache size of "/var/lib/openqa/cache" is 0 Byte, with limit 120 GiB
[info] [#17] Downloading "SLE-12-SP4-SAP-DVD-x86_64-GM-DVD1.iso" from "http://openqaworker15.qa.suse.cz/tests/5526/asset/iso/SLE-12-SP4-SAP-DVD-x86_64-GM-DVD1.iso"
[info] [#17] Cache size 960 MiB + needed 4.1 GiB exceeds limit of 120 GiB, purging least used assets
[info] [#17] Purging "/var/lib/openqa/cache/openqaworker15.qa.suse.cz/openqa_support_server_sles12sp3.x86_64.qcow2" because we need space for new assets, reclaiming 960 MiB
[info] [#17] Size of "/var/lib/openqa/cache/openqaworker15.qa.suse.cz/SLE-12-SP4-SAP-DVD-x86_64-GM-DVD1.iso" is 4.1 GiB, with ETag ""105500000-57a13ffcac780""
[info] [#17] Download of "/var/lib/openqa/cache/openqaworker15.qa.suse.cz/SLE-12-SP4-SAP-DVD-x86_64-GM-DVD1.iso" successful (630 MiB/s), new cache size is 4.1 GiB

[2022-08-17T23:49:29.068867+02:00] [info] Rsync from 'rsync://openqaworker15.qa.suse.cz/tests' to '/var/lib/openqa/cache/openqaworker15.qa.suse.cz', request #20 sent to Cache Service
[2022-08-18T00:49:18.889378+02:00] [info] +++ worker notes +++
[2022-08-18T00:49:18.889589+02:00] [info] End time: 2022-08-17 22:49:18
[2022-08-18T00:49:18.889664+02:00] [info] Result: timeout
[2022-08-18T00:49:18.898315+02:00] [info] Uploading autoinst-log.txt