action #16496
closed[tools][sprint 201711.2] display current disk space consumption of job groups
Description
There's a nice ui for editing the parameters of a job group
https://openqa.opensuse.org/admin/edit_parent_group/3
However, I cannot see how much space is currently consumed by which assets that are tracked as part of the job group. That information is hidden in the GRU logs.
Updated by lnussel almost 8 years ago
and the total sum of all size limits would be interesting too I guess to prevent overbooking of the disk :)
Updated by coolo almost 8 years ago
It's not so easy. Job groups can share assets, but their size limits apply per group. Only if an asset is out of the limit for all groups, it's deleted. And you shouldn't even come close to overbook the disk with assets - as the test results and temporary asset uploads are on the same disk.
Updated by coolo about 7 years ago
- Assignee set to mkittler
- Priority changed from Normal to High
- Target version set to Ready
We need to work on figuring out what eats our disk space. The asset list we have e.g. on osd has hundreds of assets - and it's unclear when they were used the last by what job group.
Marius, I see this as part of the asset list - the job groups of course also contain job test results and images, but the majority of space is eaten by assets.
Updated by szarate about 7 years ago
- Subject changed from display current disk space consumption of job groups to [tools][sprint 201711.2] display current disk space consumption of job groups
Updated by mkittler about 7 years ago
About the initial issue:
It would be grouping the assets by job group and show the accumulated size. And yes, by default this grouping would also include assets shared with other job groups. However, it would be possible to accumulate only exclusively used assets.
Not sure whether the grouping can be done with the help of the database (SQL GROUP BY
statement) rather than doing it manually. When having to filter out shared assets it is likely only possible to do it manually.
But the variant to filter out shared assets makes likely most sense because only exclusive assets are definite candidates for cleanup (and cleanup is the purpose and only use for the size limit).
About making this part of the assets list:
and it's unclear when they were used the last by what job group
...
I see this as part of the asset list
So the table of assets should show the last use? And 'the last use' includes - beside the date - the job group which has the job which lastly referenced it?
I also noticed that we actually save the size of each asset in the database but do not display it in the asset table in the web UI. At least that could be easily improved.
Updated by coolo about 7 years ago
You can also commit data back into the DB from the GRU tasks - that calculates a lot already anyway.
Updated by okurz about 7 years ago
- Related to action #28328: job was triggered trying to download HDD image but it's already gone added
Updated by mkittler about 7 years ago
- Status changed from New to In Progress
Updated by szarate about 7 years ago
- Target version changed from Ready to Current Sprint
Carry over to sprint 201712.1, assigning to "Current Sprint".
Updated by mkittler about 7 years ago
- Target version changed from Current Sprint to Ready
current PR: https://github.com/os-autoinst/openQA/pull/1525
Updated by mkittler about 7 years ago
- Target version changed from Ready to Current Sprint
Updated by mkittler almost 7 years ago
- Status changed from In Progress to Resolved
https://openqa.opensuse.org/admin/assets now shows the asset usage by group so the info is no longer hidden in the GRU logs and it works in production.
Note that there's also https://github.com/os-autoinst/openQA/pull/1553#issuecomment-356277916 which fixes the asset deletion from the web UI. Since this is not really requested here, I close this issue.
If anything is missing, just reopen.
Updated by szarate almost 7 years ago
- Target version changed from Current Sprint to Done