Actions
action #91347
closedcoordination #64746: [saga][epic] Scale up: Efficient handling of large storage to be able to run current tests efficiently but keep big archives of old results
coordination #80546: [epic] Scale up: Enable to store more results
[spike][timeboxed:18h] Support for archived jobs
Description
Motivation¶
See ideas in parent #80546
Suggestions¶
- add "archived" flag, check what would be needed to read job results based on archived flag
- move one test job to a different path, e.g.
/archive/…/$job
- if archived; read resultdir with configured archive path prefix, not normal resultdir prefix
- add minion job to archive job, triggered on what conditions?
- add feedback in UI "Archived job: Loading can take longer than usual"
Open questions to answer¶
- which jobs to archive? maybe boolean config: If "archive" then move important jobs to archive if they expire, else delete; or same for all
- when/how to trigger the archiving minion job?
- is it performant enough to have one minion job for one time or one minion job for every job or some bunching in the middle?
- How can admins and users control the archiving decisions?
- Is there a need to unarchive (move back to non-archived results)?
- How to monitor and cleanup from the archive?
- How does the current cleanup encounter archived results, delete them? fail?
- Should we consider "archived" == "important" after some grace time when an important job is moved to a potentially slower archive? So at the time when we trigger the cleanup for non-important jobs we also look at the important jobs and make sure that they are archived or queued for archiving (if that's a more costly process)
Actions