900GB of assets not tracked ('other' assets?)
/admin/assets knows about 3590GB, but our file system tracks 4400GB. There is /assets/tests with 10GB and there is /assets/factory/tmp with 1.3GB - the rest should be tracked. So where are the remaining 900GB?
I'm afraid it's LTP madness - other/ directory is at least ~900GB. And we have e.g. other/00678816-autoinst.xml from 2016. We may not scan this directory for untracked assets at all.
#7 Updated by andriinikitin 8 months ago
3547.72GiB tracked assets and 3828.09GiB used in /assets.
It looks the number for tracked assets is based on apparent file size, while actual file size may be much bigger, depending on number of small files.
E.g. I've copied 15 files starting with 01* from factory/other and got difference in apparent vs actual file size 136Kb vs 184Kb :
> mkdir test > cp /var/lib/openqa/factory/other/01* test/ > du -s --apparent test 136 test > du -s test 184 test > ls test/* | wc -l 15
Maybe we should calculate actual file size for assets as well (which may be tricky).
I think you forgot to mention that you did proceed. There was e.g. https://github.com/os-autoinst/openQA/pull/2460 which is merged by now.
What we received from o3 this morning:
will be deleted in -359 days … [2019-11-15T05:35:29.0505 UTC] [warn] [pid:7212] Asset other/ltp-net_stress.ipsec_udp-opensuse-Tumbleweed-ppc64le-20190305-DVD@ppc64le-with-ltp.qcow2.txt is not in any job group and will be deleted in -7929 days [2019-11-15T05:35:29.0506 UTC] [warn] [pid:7212] Asset other/ltp-fcntl-locktests-Tumbleweed-20180707 is not in any job group and will be deleted in -7929 days [2019-11-15T05:35:29.0508 UTC] [warn] [pid:7212] Asset other/ltp-controllers-opensuse-Tumbleweed-aarch64-20191009-DVD@aarch64-with-ltp-qcow2.txt is not in any job group and will be deleted in -7929 days [2019-11-15T05:35:29.0510 UTC] [warn] [pid:7212] Asset other/00683337-autoinst.xml is not in any job group and will be deleted in -7929 days [2019-11-15T05:35:29.0512 UTC] [warn] [pid:7212] Asset other/ltp-net.features-opensuse-Tumbleweed-x86_64-20190110-DVD@64bit-with-ltp.qcow2.txt is not in any job group and will be deleted in -7929 days
EDIT: and they are so far not being deleted.
- Status changed from In Progress to Resolved
- Target version changed from Current Sprint to Done
osd:/var/lib/openqa/share/factory/other has currently 243G and it seems these are properly tracked as assets. OSD states /assets has 4391G used, https://openqa.suse.de/admin/assets tracks 4044G so we are "missing" 347G from the tracking. Created #60713 for /var/lib/openqa/share/factory/tmp which leaves 344G. Given that we have many small files and I do not know how "du" and "df" calculate used space differently this could be the difference depending on the filesystem inode size or metadata. I guess we are good.