Project

General

Profile

Actions
Copy link

coordination #80546

closed

coordination #64746: [saga][epic] Scale up: Efficient handling of large storage to be able to run current tests efficiently but keep big archives of old results

[epic] Scale up: Enable to store more results

Added by okurz over 3 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
okurz
Category:
Feature requests
Target version:
Ready
Start date:
2020-08-04
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Tags:
storage, results, scale-up

Description

Ideas

  • Simple enlarge storage space for "results" on our production instances (which is also a test for scalability) -> #77890
  • Setup additional storage (server) for old results
    • Use overlayfs to make archived results appear alongside the regular results
    • Move old results at some point from the regular web UI host to the external storage (server), e.g. automatically via a Minion job, just move the resultdir to a different place, point to a different resultdir but keep the database entry itself and set a database flag "archived" or similar
    • Mark jobs with archived results as such

Subtasks 13 (0 open — 13 closed)

openQA Infrastructure - action #69577: Handle installation of the new "Storage Server"Resolvednicksinger2020-08-04

Actions
openQA Infrastructure - action #77890: [easy] Extend OSD storage space for "results" to make bug investigation and failure archeology easierResolvedokurz2020-11-14

Actions
openQA Infrastructure - action #88546: Make use of the new "Storage Server", e.g. complete OSD backupResolvedokurz

Actions
openQA Infrastructure - action #90629: administration of the new "Storage Server"Resolvednicksinger2020-08-04

Actions
action #91347: [spike][timeboxed:18h] Support for archived jobsResolvedmkittler2021-04-19

Actions
openQA Infrastructure - action #91779: Add monitoring for storage.qa.suse.deResolvedmkittler2021-04-26

Actions
action #91782: Add support for archived jobsResolvedmkittler2021-04-26

Actions
action #91785: UI representation for archived jobsResolvedmkittler

Actions
action #92788: Use openQA archiving feature on osd size:SResolvedokurz

Actions
openQA Infrastructure - coordination #96974: [epic] Improve/reconsider thresholds for skipping cleanupResolvedokurz2021-09-20

Actions
openQA Infrastructure - action #98922: Run asset cleanup concurrently to results based on configResolvedmkittler2021-09-20

Actions
openQA Infrastructure - action #103954: Run asset cleanup concurrently to results based on config on o3 as wellResolvedokurz

Actions
action #103953: Use openQA archiving feature on o3 size:SResolvedokurz2021-12-14

Actions
Actions
Copy link
 #1

Updated by okurz about 3 years ago

  • Description updated (diff)
Actions
Copy link
 #2

Updated by okurz about 3 years ago

  • Description updated (diff)
Actions
Copy link
 #3

Updated by okurz over 2 years ago

  • Status changed from Blocked to Resolved

With the archiving feature present and enabled on both o3+osd we are much more flexible and should be good for the future

Actions
Copy link

Also available in: Atom PDF