action #119215
closed[openQA][repos][aarch64][15-SP5[Full Media] 15-SP5 Full media is missing from http://openqa.suse.de/assets/repo/
Added by waynechen55 about 2 years ago. Updated about 2 years ago.
0%
Description
Observation¶
There is no 15-SP5 full media for aarch64 on website http://openqa.suse.de/assets/repo/. I do not think there is something still under building at the moment on build.suse.de for Build29.1.
Only aarch64 full media is missing from openQA website:
Steps to reproduce¶
- Go to openQA website http://openqa.suse.de/assets/repo/
- Check full media for aarch64
Impact¶
Any test suite using aarch64 full media will fail.
Problem¶
It seems there is some thing wrong with openQA infrastructure
Suggestion¶
- Check why full media build for aarch64 is missing
Workaround¶
n/a
Files
Updated by okurz about 2 years ago
- Status changed from New to In Progress
- Assignee set to okurz
Updated by okurz about 2 years ago
All important results should be visible from https://openqa.suse.de/admin/obs_rsync . Following that link I find https://openqa.suse.de/admin/obs_rsync/SUSE:SLE-15-SP5:GA:TEST%7Cbase/runs/.run_221019_103656_29.1 for build 29.1. https://openqa.suse.de/admin/obs_rsync/SUSE:SLE-15-SP5:GA:TEST%7Cbase/runs/.run_221019_103656_29.1/download/files_iso.lst lists the full medium for all four architectures. The full "repository" is extracted on the openQA host itself from the ISO file, visible in:
https://openqa.suse.de/admin/obs_rsync/SUSE:SLE-15-SP5:GA:TEST%7Cbase/runs/.run_221019_103656_29.1/download/rsync_iso.cmd
rsync --timeout=3600 -tlp4 --specials dist.suse.de::repos/SUSE:/SLE-15-SP5:/GA:/TEST/images/iso/*SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso /var/lib/openqa/factory/iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso
rsync --timeout=3600 -tlp4 --specials dist.suse.de::repos/SUSE:/SLE-15-SP5:/GA:/TEST/images/iso/*SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 /var/lib/openqa/factory/other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256
[ -d /var/lib/openqa/factory/repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 ] || {
mkdir /var/lib/openqa/factory/repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1
bsdtar xf /var/lib/openqa/factory/iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso -C /var/lib/openqa/factory/repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1
}
with the logs in https://openqa.suse.de/admin/obs_rsync/SUSE:SLE-15-SP5:GA:TEST%7Cbase/runs/.run_221019_103656_29.1/download/rsync_iso.cmd.log
+ rsync --timeout=3600 -tlp4 --specials 'dist.suse.de::repos/SUSE:/SLE-15-SP5:/GA:/TEST/images/iso/*SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso' /var/lib/openqa/factory/iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso
+ rsync --timeout=3600 -tlp4 --specials 'dist.suse.de::repos/SUSE:/SLE-15-SP5:/GA:/TEST/images/iso/*SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256' /var/lib/openqa/factory/other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256
+ '[' -d /var/lib/openqa/factory/repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 ']'
+ mkdir /var/lib/openqa/factory/repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1
+ bsdtar xf /var/lib/openqa/factory/iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso -C /var/lib/openqa/factory/repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1
so it looks like the repo was created. Who can remind me where we can find log messages regarding the cleanup of assets?
Updated by okurz about 2 years ago
- Due date set to 2022-11-11
- Status changed from In Progress to Feedback
I found on OSD
openqa:/var/log # xzgrep -i SLE-15-SP5-Full-aarch64-Build29 openqa_gru.8.xz
[2022-10-20T04:08:31.562809+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 110 (20620640697)
[2022-10-20T04:08:31.562875+02:00] [debug] [pid:31541] Asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) picked into group 110
[2022-10-20T04:08:31.562928+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 129 (11005337889)
[2022-10-20T04:08:31.563014+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 143 (163476307)
[2022-10-20T04:08:31.563077+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 251 (3067777832)
[2022-10-20T04:08:31.563148+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 265 (400818176)
[2022-10-20T04:08:31.563241+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 267 (5771211776)
[2022-10-20T04:08:31.563302+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 301 (3321888768)
[2022-10-20T04:08:31.563357+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 452 (46858240)
[2022-10-20T04:08:31.563710+02:00] [debug] [pid:31541] Checking whether asset repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 (12108819405) fits into group 110 (10068957625)
[2022-10-20T04:08:31.563781+02:00] [debug] [pid:31541] Checking whether asset repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 (12108819405) fits into group 129 (11005337889)
[2022-10-20T04:08:31.563840+02:00] [debug] [pid:31541] Checking whether asset repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 (12108819405) fits into group 143 (163476307)
[2022-10-20T04:08:31.563910+02:00] [debug] [pid:31541] Checking whether asset repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 (12108819405) fits into group 267 (5771211776)
[2022-10-20T04:08:31.563980+02:00] [debug] [pid:31541] Checking whether asset repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 (12108819405) fits into group 452 (46858240)
[2022-10-20T04:08:33.396420+02:00] [debug] [pid:31541] Checking whether asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) fits into group 110 (1590165945)
[2022-10-20T04:08:33.396470+02:00] [debug] [pid:31541] Asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) picked into group 110
[2022-10-20T04:08:33.396522+02:00] [debug] [pid:31541] Checking whether asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) fits into group 129 (445919993)
[2022-10-20T04:08:33.396573+02:00] [debug] [pid:31541] Checking whether asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) fits into group 143 (17576275)
[2022-10-20T04:08:33.396623+02:00] [debug] [pid:31541] Checking whether asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) fits into group 452 (46196224)
name => "iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso",
name => "repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1",
name => "other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256",
[2022-10-20T04:18:45.478011+02:00] [info] [pid:31541] Removing asset repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 (belonging to job groups: 129, 143, 452, 267, 110 within parent job groups 15)
[2022-10-20T04:19:33.121798+02:00] [info] [pid:31541] GRU: removed /var/lib/openqa/share/factory/repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1
[2022-10-20T04:28:59.262163+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 110 (20609593380)
[2022-10-20T04:28:59.262233+02:00] [debug] [pid:31541] Asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) picked into group 110
[2022-10-20T04:28:59.262310+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 129 (11005337889)
[2022-10-20T04:28:59.262407+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 143 (163476307)
[2022-10-20T04:28:59.262494+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 251 (3067777832)
[2022-10-20T04:28:59.262572+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 265 (400818176)
[2022-10-20T04:28:59.262643+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 267 (5771211776)
[2022-10-20T04:28:59.263746+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 301 (3321888768)
[2022-10-20T04:28:59.263840+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 452 (46858240)
[2022-10-20T04:29:00.041376+02:00] [debug] [pid:31541] Checking whether asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) fits into group 110 (1579118628)
[2022-10-20T04:29:00.041454+02:00] [debug] [pid:31541] Asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) picked into group 110
[2022-10-20T04:29:00.041539+02:00] [debug] [pid:31541] Checking whether asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) fits into group 129 (445919993)
[2022-10-20T04:29:00.041624+02:00] [debug] [pid:31541] Checking whether asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) fits into group 143 (17576275)
[2022-10-20T04:29:00.041709+02:00] [debug] [pid:31541] Checking whether asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) fits into group 452 (46196224)
in particular:
[2022-10-20T04:18:45.478011+02:00] [info] [pid:31541] Removing asset repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 (belonging to job groups: 129, 143, 452, 267, 110 within parent job groups 15)
[2022-10-20T04:19:33.121798+02:00] [info] [pid:31541] GRU: removed /var/lib/openqa/share/factory/repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1
so the asset was simply removed because it does not fit anymore.
@waynechen55 Please understand how the asset management on openQA works:
http://open.qa/docs/#_asset_handling
Based on available space I could extend the size limit for assets in the functional job group (id 110) from 260GB to 400GB which should allow to save all four Full repos though still taking a lot of space.
The next build should again have a full aarch64 repo.
Further information: https://monitor.qa.suse.de/d/WebuiDb/webui-summary?orgId=1&viewPanel=74&from=now-2y&to=now shows the evolution of asset space usage over the course of the past two years. Within https://monitor.qa.suse.de/d/WebuiDb/webui-summary?orgId=1&viewPanel=74&from=now-7d&to=now we see how the space usage goes down to 74% corresponding to (0.74 * 5.0T = 5.18TB) until we reach the limit of 80% (0.80 * 5.0T = 5.60TB) when we trigger the space-aware cleanup. So effectively we have a buffer of 5.60TB - 5.18TB = 0.420 ~ 420GB which we can still assign to groups.
If you need much more space for assets on openqa.suse.de please organize with all relevant user groups to formulate such request to SUSE-IT.
Updated by okurz about 2 years ago
- Related to action #73183: Extend vdc (/assets) added
Updated by okurz about 2 years ago
- Related to action #71575: [osd][alert] limited /assets - idea: ask EngInfra for slow+cheap storage from central server for /assets/fixed only added
Updated by okurz about 2 years ago
I opened https://sd.suse.com/servicedesk/customer/portal/1/SD-102307
Kind request to extend vdc assigned to VM openqa.suse.de from 7.0T to 10.0T Waiting for support
This is a kind request to extend vdc assigned to VM openqa.suse.de from 7.0T to 10.0T which should reside on “cheap+slow” storage, i.e. rotating disks. Reasoning: More and more products are tested by more and more development groups hence demanding more space. Related internal issue: https://progress.opensuse.org/issues/119215
Updated by waynechen55 about 2 years ago
okurz wrote:
I found on OSD
openqa:/var/log # xzgrep -i SLE-15-SP5-Full-aarch64-Build29 openqa_gru.8.xz [2022-10-20T04:08:31.562809+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 110 (20620640697) [2022-10-20T04:08:31.562875+02:00] [debug] [pid:31541] Asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) picked into group 110 [2022-10-20T04:08:31.562928+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 129 (11005337889) [2022-10-20T04:08:31.563014+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 143 (163476307) [2022-10-20T04:08:31.563077+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 251 (3067777832) [2022-10-20T04:08:31.563148+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 265 (400818176) [2022-10-20T04:08:31.563241+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 267 (5771211776) [2022-10-20T04:08:31.563302+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 301 (3321888768) [2022-10-20T04:08:31.563357+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 452 (46858240) [2022-10-20T04:08:31.563710+02:00] [debug] [pid:31541] Checking whether asset repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 (12108819405) fits into group 110 (10068957625) [2022-10-20T04:08:31.563781+02:00] [debug] [pid:31541] Checking whether asset repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 (12108819405) fits into group 129 (11005337889) [2022-10-20T04:08:31.563840+02:00] [debug] [pid:31541] Checking whether asset repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 (12108819405) fits into group 143 (163476307) [2022-10-20T04:08:31.563910+02:00] [debug] [pid:31541] Checking whether asset repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 (12108819405) fits into group 267 (5771211776) [2022-10-20T04:08:31.563980+02:00] [debug] [pid:31541] Checking whether asset repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 (12108819405) fits into group 452 (46858240) [2022-10-20T04:08:33.396420+02:00] [debug] [pid:31541] Checking whether asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) fits into group 110 (1590165945) [2022-10-20T04:08:33.396470+02:00] [debug] [pid:31541] Asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) picked into group 110 [2022-10-20T04:08:33.396522+02:00] [debug] [pid:31541] Checking whether asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) fits into group 129 (445919993) [2022-10-20T04:08:33.396573+02:00] [debug] [pid:31541] Checking whether asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) fits into group 143 (17576275) [2022-10-20T04:08:33.396623+02:00] [debug] [pid:31541] Checking whether asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) fits into group 452 (46196224) name => "iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso", name => "repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1", name => "other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256", [2022-10-20T04:18:45.478011+02:00] [info] [pid:31541] Removing asset repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 (belonging to job groups: 129, 143, 452, 267, 110 within parent job groups 15) [2022-10-20T04:19:33.121798+02:00] [info] [pid:31541] GRU: removed /var/lib/openqa/share/factory/repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 [2022-10-20T04:28:59.262163+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 110 (20609593380) [2022-10-20T04:28:59.262233+02:00] [debug] [pid:31541] Asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) picked into group 110 [2022-10-20T04:28:59.262310+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 129 (11005337889) [2022-10-20T04:28:59.262407+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 143 (163476307) [2022-10-20T04:28:59.262494+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 251 (3067777832) [2022-10-20T04:28:59.262572+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 265 (400818176) [2022-10-20T04:28:59.262643+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 267 (5771211776) [2022-10-20T04:28:59.263746+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 301 (3321888768) [2022-10-20T04:28:59.263840+02:00] [debug] [pid:31541] Checking whether asset iso/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso (10551683072) fits into group 452 (46858240) [2022-10-20T04:29:00.041376+02:00] [debug] [pid:31541] Checking whether asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) fits into group 110 (1579118628) [2022-10-20T04:29:00.041454+02:00] [debug] [pid:31541] Asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) picked into group 110 [2022-10-20T04:29:00.041539+02:00] [debug] [pid:31541] Checking whether asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) fits into group 129 (445919993) [2022-10-20T04:29:00.041624+02:00] [debug] [pid:31541] Checking whether asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) fits into group 143 (17576275) [2022-10-20T04:29:00.041709+02:00] [debug] [pid:31541] Checking whether asset other/SLE-15-SP5-Full-aarch64-Build29.1-Media1.iso.sha256 (111) fits into group 452 (46196224)in particular:
[2022-10-20T04:18:45.478011+02:00] [info] [pid:31541] Removing asset repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1 (belonging to job groups: 129, 143, 452, 267, 110 within parent job groups 15) [2022-10-20T04:19:33.121798+02:00] [info] [pid:31541] GRU: removed /var/lib/openqa/share/factory/repo/SLE-15-SP5-Full-aarch64-Build29.1-Media1so the asset was simply removed because it does not fit anymore.
@waynechen55 Please understand how the asset management on openQA works:
http://open.qa/docs/#_asset_handlingBased on available space I could extend the size limit for assets in the functional job group (id 110) from 260GB to 400GB which should allow to save all four Full repos though still taking a lot of space.
The next build should again have a full aarch64 repo.
Further information: https://monitor.qa.suse.de/d/WebuiDb/webui-summary?orgId=1&viewPanel=74&from=now-2y&to=now shows the evolution of asset space usage over the course of the past two years. Within https://monitor.qa.suse.de/d/WebuiDb/webui-summary?orgId=1&viewPanel=74&from=now-7d&to=now we see how the space usage goes down to 74% corresponding to (0.74 * 5.0T = 5.18TB) until we reach the limit of 80% (0.80 * 5.0T = 5.60TB) when we trigger the space-aware cleanup. So effectively we have a buffer of 5.60TB - 5.18TB = 0.420 ~ 420GB which we can still assign to groups.
If you need much more space for assets on openqa.suse.de please organize with all relevant user groups to formulate such request to SUSE-IT.
I can not have no 15-SP5 aarch64 Builds for testing. It is not something can be sacrificed.
Updated by waynechen55 about 2 years ago
okurz wrote:
I opened https://sd.suse.com/servicedesk/customer/portal/1/SD-102307
Kind request to extend vdc assigned to VM openqa.suse.de from 7.0T to 10.0T Waiting for support
This is a kind request to extend vdc assigned to VM openqa.suse.de from 7.0T to 10.0T which should reside on “cheap+slow” storage, i.e. rotating disks. Reasoning: More and more products are tested by more and more development groups hence demanding more space. Related internal issue: https://progress.opensuse.org/issues/119215
Thanks for your support @okurz. I hope the next build which will be released on 2022-10-27 has full media for aarch64. Otherwise it impacts 15-SP5 final alpha QA full run.
Updated by okurz about 2 years ago
As alternative you could trigger a resync and trigger openQA tests but this would be multiple hundreds of jobs. What could also work is to manually extract the repo folder and retrigger only the relevant tests
Updated by okurz about 2 years ago
- Due date deleted (
2022-11-11) - Status changed from Feedback to Resolved
https://sd.suse.com/servicedesk/customer/portal/1/SD-102307 resolved. I increased the partition and filesystem
okurz@openqa:~> sudo parted -s -a opt /dev/vdc resizepart 1 100%
Warning: Partition /dev/vdc is being used, continuing anyway.
okurz@openqa:~> sudo xfs_growfs /dev/vdc
meta-data=/dev/vdc isize=512 agcount=7, agsize=268435455 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0, rmapbt=0
= reflink=1 bigtime=0 inobtcount=0
data = bsize=4096 blocks=1879048185, imaxpct=5
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1
log =internal log bsize=4096 blocks=521728, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
data blocks changed from 1879048185 to 2684354550
okurz@openqa:~> df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 4.0M 8.0K 4.0M 1% /dev
tmpfs 12G 3.6M 12G 1% /dev/shm
tmpfs 4.7G 1.2M 4.7G 1% /run
tmpfs 4.0M 0 4.0M 0% /sys/fs/cgroup
/dev/vda1 20G 6.8G 12G 37% /
/dev/vdb 200G 58G 143G 29% /srv
/dev/vde 5.5T 3.2T 2.4T 58% /space-slow
/dev/vdc 10T 5.6T 4.5T 56% /assets
/dev/loop0 4.9G 1.3G 3.4G 27% /home
/dev/vdd 5.5T 3.9T 1.6T 71% /results
tmpfs 2.0G 0 2.0G 0% /run/user/1001
tmpfs 2.0G 0 2.0G 0% /run/user/0
tmpfs 2.0G 0 2.0G 0% /run/user/17254
I bumped up some asset quota limits in job groups. That should be more than enough space for now.
Updated by waynechen55 about 2 years ago
okurz wrote:
https://sd.suse.com/servicedesk/customer/portal/1/SD-102307 resolved. I increased the partition and filesystem
okurz@openqa:~> sudo parted -s -a opt /dev/vdc resizepart 1 100% Warning: Partition /dev/vdc is being used, continuing anyway. okurz@openqa:~> sudo xfs_growfs /dev/vdc meta-data=/dev/vdc isize=512 agcount=7, agsize=268435455 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=0, rmapbt=0 = reflink=1 bigtime=0 inobtcount=0 data = bsize=4096 blocks=1879048185, imaxpct=5 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=521728, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 data blocks changed from 1879048185 to 2684354550 okurz@openqa:~> df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 4.0M 8.0K 4.0M 1% /dev tmpfs 12G 3.6M 12G 1% /dev/shm tmpfs 4.7G 1.2M 4.7G 1% /run tmpfs 4.0M 0 4.0M 0% /sys/fs/cgroup /dev/vda1 20G 6.8G 12G 37% / /dev/vdb 200G 58G 143G 29% /srv /dev/vde 5.5T 3.2T 2.4T 58% /space-slow /dev/vdc 10T 5.6T 4.5T 56% /assets /dev/loop0 4.9G 1.3G 3.4G 27% /home /dev/vdd 5.5T 3.9T 1.6T 71% /results tmpfs 2.0G 0 2.0G 0% /run/user/1001 tmpfs 2.0G 0 2.0G 0% /run/user/0 tmpfs 2.0G 0 2.0G 0% /run/user/17254I bumped up some asset quota limits in job groups. That should be more than enough space for now.
Thanks for your timely support.