Project

General

Profile

coordination #80546

coordination #64746: [saga][epic] Scale up: Efficient handling of large storage to be able to run current tests efficiently but keep big archives of old results

[epic] Scale up: Enable to store more results

Added by okurz over 1 year ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2020-08-04
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

Ideas

  • Simple enlarge storage space for "results" on our production instances (which is also a test for scalability) -> #77890
  • Setup additional storage (server) for old results
    • Use overlayfs to make archived results appear alongside the regular results
    • Move old results at some point from the regular web UI host to the external storage (server), e.g. automatically via a Minion job, just move the resultdir to a different place, point to a different resultdir but keep the database entry itself and set a database flag "archived" or similar
    • Mark jobs with archived results as such

Subtasks

openQA Infrastructure - action #69577: Handle installation of the new "Storage Server"Resolvednicksinger

openQA Infrastructure - action #77890: [easy] Extend OSD storage space for "results" to make bug investigation and failure archeology easierResolvedokurz

openQA Infrastructure - action #88546: Make use of the new "Storage Server", e.g. complete OSD backupResolvedokurz

openQA Infrastructure - action #90629: administration of the new "Storage Server"Resolvednicksinger

action #91347: [spike][timeboxed:18h] Support for archived jobsResolvedmkittler

openQA Infrastructure - action #91779: Add monitoring for storage.qa.suse.deResolvedmkittler

action #91782: Add support for archived jobsResolvedmkittler

action #91785: UI representation for archived jobsResolvedmkittler

action #92788: Use openQA archiving feature on osd size:SResolvedokurz

openQA Infrastructure - coordination #96974: [epic] Improve/reconsider thresholds for skipping cleanupResolvedokurz

openQA Infrastructure - action #98922: Run asset cleanup concurrently to results based on configResolvedmkittler

openQA Infrastructure - action #103954: Run asset cleanup concurrently to results based on config on o3 as wellResolvedokurz

action #103953: Use openQA archiving feature on o3 size:SResolvedokurz

History

#1 Updated by okurz about 1 year ago

  • Description updated (diff)

#2 Updated by okurz about 1 year ago

  • Description updated (diff)

#3 Updated by okurz 5 months ago

  • Status changed from Blocked to Resolved

With the archiving feature present and enabled on both o3+osd we are much more flexible and should be good for the future

Also available in: Atom PDF