Project

General

Profile

coordination #80546

coordination #64746: [saga][epic] Scale up: Efficient handling of large storage to be able to run current tests efficiently but keep big archives of old results

[epic] Scale up: Enable to store more results

Added by okurz about 1 year ago. Updated 29 days ago.

Status:
Blocked
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Start date:
2020-08-04
Due date:
% Done:

75%

Estimated time:
(Total: 0.00 h)
Difficulty:

Description

Ideas

  • Simple enlarge storage space for "results" on our production instances (which is also a test for scalability) -> #77890
  • Setup additional storage (server) for old results
    • Use overlayfs to make archived results appear alongside the regular results
    • Move old results at some point from the regular web UI host to the external storage (server), e.g. automatically via a Minion job, just move the resultdir to a different place, point to a different resultdir but keep the database entry itself and set a database flag "archived" or similar
    • Mark jobs with archived results as such

Subtasks

openQA Infrastructure - action #69577: Handle installation of the new "Storage Server"Resolvednicksinger

openQA Infrastructure - action #77890: [easy] Extend OSD storage space for "results" to make bug investigation and failure archeology easierResolvedokurz

openQA Infrastructure - action #88546: Make use of the new "Storage Server", e.g. complete OSD backupBlockedokurz

openQA Infrastructure - action #90629: administration of the new "Storage Server"Resolvednicksinger

action #91347: [spike][timeboxed:18h] Support for archived jobsResolvedmkittler

openQA Infrastructure - action #91779: Add monitoring for storage.qa.suse.deResolvedmkittler

action #91782: Add support for archived jobsResolvedmkittler

action #91785: UI representation for archived jobsResolvedmkittler

action #92788: Use openQA archiving feature on osd size:SBlockedokurz

openQA Infrastructure - action #96269: Define what a "complete OSD backup" should or can includeBlockedokurz

openQA Infrastructure - coordination #96974: [epic] Improve/reconsider thresholds for skipping cleanupNew

openQA Infrastructure - action #98922: Run asset cleanup concurrently to results based on configResolvedmkittler

History

#1 Updated by okurz 8 months ago

  • Description updated (diff)

#2 Updated by okurz 8 months ago

  • Description updated (diff)

Also available in: Atom PDF