Project

General

Profile

coordination #91646

[saga][epic] SUSE Maintenance QA workflows with fully automated testing, approval and release

Added by okurz about 1 year ago. Updated 1 day ago.

Status:
Blocked
Priority:
Normal
Assignee:
Target version:
Start date:
2021-04-21
Due date:
2022-05-18
% Done:

92%

Estimated time:
(Total: 0.00 h)

Description

Motivation

Scale up the business while ensuring highest quality with automated testing, approval and release of package updates for SLE and openSUSE


Subtasks

openQA Project - coordination #89062: [epic] Simplify review for SUSE QAMResolvedokurz

openQA Project - action #91542: openQA API what jobs were/are testing X incident/packageResolvedilausuch

openQA Project - action #91658: Make "black certificate" stricter to only show when /tests/overview?todo=1 is empty, i.e. no unlabeled failuresResolvedmkittler

openQA Project - action #91773: Automatic replacement of openQA job URLs preview of openQA size:MResolvedtinita

openQA Infrastructure - action #92034: Re-enable openqa-investigate options after the black certificate now only shows properly "reviewed" jobsResolvedmkittler

action #92851: Workshop series proposal "How SUSE QE teams review openQA test results"Resolvedcdywan

openQA Project - coordination #93246: [epic] List all unreviewed failed (or incomplete) jobs on /tests on request size:MResolvedokurz

openQA Project - action #103765: Support for "todo" query parameter on /tests, same as /tests/overview size:MResolvedtinita

openQA Project - action #104995: Add UI element and help text for "todo" query parameter on /tests, similar as /tests/overviewResolvedmkittler

openQA Project - action #105001: Add doc for "todo" query parameter on /tests, similar to /tests/overview size:SResolvedkodymo

openQA Project - action #94937: Distinguish comment types on jobs on /tests (maybe optional) size:SResolvedtinita

openQA Project - action #102374: Support use of force_result via ticket title in auto-review size:MResolvedtinita

openQA Project - openqa-force-result #109857: Secure auto-review+force_result size:M auto_review:"Failed to download gobbledeegoop":force_result:softfailedResolvedcdywan
action #107014: trigger openqa-trigger-bisect-jobs from our automatic investigations whenever the cause is not already known size:MResolvedtinita

openQA Project - coordination #91467: [epic] Surface openQA failures per squad in a single placeResolvedokurz

openQA Project - action #91647: Making option to filter by flavor, test name on /tests/overview more prominentResolvedkraih

openQA Project - action #91650: Resolve the most recent builds per job group on /tests/overview when showing multiple job groupsResolvedilausuch

openQA Project - action #91652: Remind about the use of openqa-review in squadsResolvedokurz

openQA Project - action #92957: Add option to openqa-review to skip displaying all passed resultsResolvedtinita

openQA Project - action #93727: Publish openqa-review reports with "--skip-passed"Resolvedokurz

openQA Project - action #94732: Provide link to /tests/overview of latest builds of all job groups within a parent job group size:MResolvedilausuch

openQA Project - action #94762: openqa-review: Add mode of single-line todo lists size:MResolvedtinita

openQA Project - action #96058: [spike] Filter test results on /tests or /tests/overview by regex match in modules size:MResolvedosukup

action #97403: openqa-review: Polish job group section titles in todo-only mode size:SResolvedkodymo

openQA Project - action #98258: No results on /tests/overview w/o buildResolvedosukup

openQA Project - action #98445: improve description for "Test module" UI element as followup to #96058Resolvedosukup

openQA Project - action #98460: Filter actual test results on /tests or /tests/overview by regex match in modulesResolvedokurz

openQA Project - coordination #91914: [epic] Make reviewing openQA results per squad easierBlockedokurz

openQA Project - action #91605: notifications about failed and unreviewed jobs - but using Slack (was: Rocket.Chat) size:MResolvedtinita

openQA Project - action #93065: "parallel_failed" jobs show up on /tests/overview?todo=1 but these do not need a labelResolvedokurz

openQA Project - action #110629: openqa-label-known-issues: Fallback notification address in openqa-label-known-issues if no email address could be parsed from group_overviewResolvedtinita

openQA Project - action #111066: Document suggested workflows for multiple teams reviewing openQA test resultsNew

openQA Project - action #105382: Reconsider coloring of "failed modules" to make it obvious that they actually failResolvedokurz

action #107923: qem-bot: Ignore not-ok openQA jobs for specific incident based on openQA job commentBlockedokurz

coordination #98457: [epic] Handle openqa-review reminder comments on very old jobs betterResolvedokurz

action #100982: openqa-review: Do not post reminder comment if comment would be exactly the same as the last size:MResolvedjbaier_cz

action #101722: openqa-review: Do not post reminder comment in progress.opensuse.org if comment would be exactly the same as the last size:MResolvedtinita

action #106368: openqa-review: Configurable no-reminder message pattern size:MResolvedjbaier_cz

action #106907: Exponential backoff for reminders based on previous reminders size:MResolvedjbaier_cz

action #106909: [openqa-review] reminder comments point to specific openQA test details steps or openQA comments size:MResolvedjbaier_cz

action #103701: Resubmited incident (ID) with new release request (RR) inherits incident test results from previous RRResolvedosukup

openQA Project - action #104199: Prevent confusion when openQA comments look like both a bugref as well as label at the same time size:MResolvedkraih

openQA Project - action #104616: Make openQA labels clearly visibleResolvedkraih

action #108869: Missing (re-)schedules of SLE maintenance tests size:MResolvedosukup

action #108944: 5 whys follow-up to Missing (re-)schedules of SLE maintenance tests size:MResolvedosukup

coordination #109641: [epic] qem-bot improvementsBlockedokurz

action #106179: No aggregate maintenance runs scheduled today on osd - dashboard.qem.suse.de down size:SResolvedosukup

openQA Project - action #109310: qem-bot/dashboard - mixed old and new incidents size:MResolvedkraih

action #109488: qem-bot - better loggingResolvedosukup

action #109512: qem-bot - add vars with GitlabCI job link and qem-dashboard linkResolvedmkittler

action #109623: Allow adding scheduling settings for informal purposes that are not added to openQA jobsResolvedmkittler

action #109977: qem-bot - approve pipeline failed with 403 forbidden size:MResolvedokurz

action #110167: Tests for qem-botResolvedokurz

action #110409: qem-dashboard - remove old openQA jobs when rr_number changes size:MResolvedkraih

action #111075: Collect code coverage for qem-botResolvedokurz

action #111078: Simple automatic test exercising one of the existing happy path workflows of qem-botNew

action #109779: Cannot approve incident due to test report parsing errorResolvedjbaier_cz

coordination #109818: [epic] qa-maintenance/openQAbot improvementsResolvedokurz

openQA Project - action #109815: Add retry for HTTP requests in openQABot where missingResolvedjbaier_cz


Related issues

Copied to QA - coordination #99303: [saga][epic] Future improvements for SUSE Maintenance QA workflows with fully automated testing, approval and releaseNew2016-02-092021-12-18

History

#1 Updated by okurz about 1 year ago

  • Project changed from SUSE QA to QA

#3 Updated by okurz about 1 year ago

  • Tracker changed from action to coordination

#4 Updated by okurz about 1 year ago

  • Subject changed from [saga][epic] Improve SUSE Maintenance QA workflows with fully automated testing, approval and release to [saga][epic] SUSE Maintenance QA workflows with fully automated testing, approval and release

#5 Updated by okurz about 1 year ago

  • Target version changed from Ready to future

with #91605 put to future again, see #91605#note-4 , with the support of more internal stakeholders, with the need of focussing on existing sagas to get something done before we pick up more new stories, with the quick solutions applied as part of #91542 and #91467 we decide that currently for the foreseeable future of next weeks we will not continue work here and focus on other sagas instead, see https://progress.opensuse.org/issues?query_id=524

#7 Updated by okurz 11 months ago

  • Target version changed from future to Ready

#9 Updated by okurz 8 months ago

  • Copied to coordination #99303: [saga][epic] Future improvements for SUSE Maintenance QA workflows with fully automated testing, approval and release added

#10 Updated by okurz 5 months ago

Had a discussion with vpelcak about some topics:

  • mawerner has stated again that "gut-feeling" is the base for product validation -> direct consequence is that maintenance tests will fail and nobody cares to fix them
  • proposal to move more tests from aggregate to incident tests
  • I think that the cost-benefit ratio investing effort in automatic classification #102915 is better than for #104209

Also available in: Atom PDF