Project

General

Profile

Actions

coordination #92323

open

[saga][epic] Scale up: Fine-grained control over use and removal of results, assets, test data

Added by okurz almost 4 years ago. Updated 10 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Feature requests
Target version:
Start date:
2020-05-20
Due date:
% Done:

35%

Estimated time:
(Total: 0.00 h)

Description

Ideas

  • Have a configurable list of file type/name/pattern with retention period or size quota for each file type/name/pattern
  • Allow to configure retention period for the video individually #67087
  • Something better than single fixed, unflexible volumes. E.g. dm-cache, lvmcache, bcache, SUSE enterprise storage. Maybe also splitting stored results in openQA by "recent, active" and "old, archived" and then put both categories in different folders which can be mounted from different storage locations, e.g. fast, expensive for "recent, active" and slow, cheap, big for "old, archived"
  • try out tuned ext4 for worker pool fs to use a more modern default base fs than ext2 even though internally it's the same new code but we might benefit still, e.g. from faster fs formatting times. #64746#note-2

Subtasks 13 (10 open3 closed)

action #67087: Allow to configure retention period for the video individuallyNew2020-05-20

Actions
openQA Infrastructure (public) - action #77845: Use external videoencoder in production on all osd machines size:MResolvedokurz2020-11-13

Actions
openQA Infrastructure (public) - action #77848: No more workarounds in OSD due to inefficient video encoder size:MResolvedokurz2020-11-13

Actions
coordination #160302: [epic] Ensure no job results are present in the file system for jobs that are no longer in the databaseNew

Actions
QA (public) - action #134888: Ensure no job results are present in the file system for jobs that are no longer in the database - on OSD onlyNew

Actions
coordination #179221: [epic] Support keeping only jobs in database, remove all logs, assets, test results soonerNew2025-03-19

Actions
coordination #179224: Support keeping only jobs in database, remove all logs, assets, test results soonerNew2025-03-19

Actions
action #179263: [spike][timeboxed:10h] Support keeping only jobs in database, remove all logs, assets, test results soonerNew2025-03-20

Actions
coordination #179888: [epic] Creating, tracking, accounting "supporting jobs"New2024-08-06

Actions
action #164988: Better accounting for openqa-investigation jobs size:SResolvedybonatakis2024-08-06

Actions
action #179894: [spike][timeboxed:10h] Count assets+results of openqa-investigate jobs towards the originating groupNew

Actions
coordination #179891: [epic] More fine-grained asset quota controlNew2025-03-18

Actions
action #179095: Unable to set an asset quota of zeroNew2025-03-18

Actions

Related issues 1 (0 open1 closed)

Copied from openQA Project (public) - coordination #64746: [saga][epic] Scale up: Efficient handling of large storage to be able to run current tests efficiently but keep big archives of old resultsResolvedokurz2020-03-18

Actions
Actions #1

Updated by okurz almost 4 years ago

  • Copied from coordination #64746: [saga][epic] Scale up: Efficient handling of large storage to be able to run current tests efficiently but keep big archives of old results added
Actions #2

Updated by okurz 11 months ago

  • Subtask #160302 added
Actions #3

Updated by okurz 6 months ago

  • Subtask #164988 added
Actions #4

Updated by okurz 25 days ago

  • Subtask #179095 added
Actions #5

Updated by okurz 23 days ago

  • Subtask #179221 added
Actions #6

Updated by okurz 10 days ago

  • Subtask #179888 added
Actions #7

Updated by okurz 10 days ago

  • Subtask #179891 added
Actions #8

Updated by okurz 10 days ago

  • Subtask deleted (#179095)
Actions #9

Updated by okurz 10 days ago

  • Subtask deleted (#164988)
Actions

Also available in: Atom PDF