Project

General

Profile

openqa-force-result #109857

QA - coordination #91646: [saga][epic] SUSE Maintenance QA workflows with fully automated testing, approval and release

coordination #89062: [epic] Simplify review for SUSE QAM

Secure auto-review+force_result size:M auto_review:"Failed to download gobbledeegoop":force_result:softfailed

Added by okurz 3 months ago. Updated about 1 month ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Due date:

Description

Motivation

See #104923 . We should only allow a trusted set of people to influence test results on o3+osd using force_result in auto-review tickets.

  • AC1: Only a trusted, known set of people can influence test results on o3+osd using force_result in auto-review tickets

Suggestions

  • Define a new "tracker" in progress.opensuse.org (we already use "action" and "coordination", could add another one there, e.g. "openqa-force-result")
  • Configure "Non member"-permissions to disallow create+update of tickets.
  • Create new sub-project of openqatests using only that tracker (where we trust members, e.g. the "QA" project tree)
  • Extend openqa-label-known-issues to only look into that project for auto-review+force-result (optionally only the specific tracker)

This paired with a progress project where we trust members plus openqa-label-known-issues to only look at a certain tracker for "openqa-force-result" tickets would mean that only members of the specified project can create+update "openqa-force-result" tickets in the "openqa-force-result" tracker.

Rejected alternative: Create a private ticket but this has the significant drawback that soon openQA jobs will be labeled with tickets automatically and not everybody can see the tickets


Related issues

Related to QA - action #107923: qem-bot: Ignore not-ok openQA jobs for specific incident based on openQA job commentNew

History

#1 Updated by okurz 3 months ago

  • Parent task set to #104923

#2 Updated by cdywan 2 months ago

  • Subject changed from Secure auto-review+force_result to Secure auto-review+force_result size:M
  • Status changed from New to Workable

#3 Updated by cdywan about 2 months ago

  • Status changed from Workable to In Progress
  • Assignee set to cdywan

I'll give this a go. Maybe as a side effect this will teach me some new Readmine foo

#4 Updated by cdywan about 2 months ago

I created openQA investigation findings which uses the also newly created openqa-force-result and removed the ability for Non member to add or delete tickets or notes.

My user who isn't part of any project now receives the following message when trying to file a ticket: The project doesn't have any trackers for which you can create an issue.

#5 Updated by openqa_review about 2 months ago

  • Due date set to 2022-05-18

Setting due date based on mean cycle time of SUSE QE Tools

#7 Updated by okurz about 2 months ago

cdywan wrote:

I created openQA investigation findings which uses the also newly created openqa-force-result and removed the ability for Non member to add or delete tickets or notes.

My user who isn't part of any project now receives the following message when trying to file a ticket: The project doesn't have any trackers for which you can create an issue.

I consider https://progress.opensuse.org/projects/openqa-investigation-findings/ rather long and also I am afraid that it will mislead people to report tickets in there which should be in "openqatests" instead. How about "openqa-auto-review"?

#8 Updated by cdywan about 2 months ago

okurz wrote:

cdywan wrote:

I created openQA investigation findings which uses the also newly created openqa-force-result and removed the ability for Non member to add or delete tickets or notes.

My user who isn't part of any project now receives the following message when trying to file a ticket: The project doesn't have any trackers for which you can create an issue.

I consider https://progress.opensuse.org/projects/openqa-investigation-findings/ rather long and also I am afraid that it will mislead people to report tickets in there which should be in "openqatests" instead. How about "openqa-auto-review"?

This is a short and readable URL in my book, but I really don't care so I'll rename it

#9 Updated by cdywan about 2 months ago

  • Status changed from In Progress to Feedback

cdywan wrote:

okurz wrote:

cdywan wrote:

I created openQA investigation findings which uses the also newly created openqa-force-result and removed the ability for Non member to add or delete tickets or notes.

My user who isn't part of any project now receives the following message when trying to file a ticket: The project doesn't have any trackers for which you can create an issue.

I consider https://progress.opensuse.org/projects/openqa-investigation-findings/ rather long and also I am afraid that it will mislead people to report tickets in there which should be in "openqatests" instead. How about "openqa-auto-review"?

This is a short and readable URL in my book, but I really don't care so I'll rename it

openqa-auto-review it is.

And only honoring force result expressions with the new tracker would seem to work regardless of the project. So at this point we can use it or stick to projects we were already using.

#10 Updated by okurz about 2 months ago

So what are your next steps planned? I suggest you actually test this

#11 Updated by cdywan about 2 months ago

okurz wrote:

So what are your next steps planned? I suggest you actually test this

Please provide constructive comments where possible by e.g. responding to my point on not needing the sub-project or making concrete suggestions like "Have you tested this yet? How about moving a ticket to the new tracker and using force_result: with a test job".

#12 Updated by cdywan about 2 months ago

  • Subject changed from Secure auto-review+force_result size:M to Secure auto-review+force_result size:M auto_review:"Failed to download gobbledeegoop":force_result:softfailed
openqa-clone-job --within-instance https://openqa.opensuse.org/t2330754 TEST=eggs RETRY=0 HDD_1=gobbledeegoop.qcow2 _GROUP=0

#13 Updated by cdywan about 2 months ago

  • Tracker changed from action to openqa-force-result
  • Category deleted (Feature requests)
  • Start date deleted (2022-04-12)

#14 Updated by cdywan about 2 months ago

Apparently this was matching "something" else before. And this also had to be re-enabled in the config on o3. Let's see if this will work: https://openqa.opensuse.org/tests/2331276

#15 Updated by okurz about 2 months ago

  • Category set to Feature requests

#16 Updated by cdywan about 2 months ago

cdywan wrote:

Apparently this was matching "something" else before. And this also had to be re-enabled in the config on o3. Let's see if this will work: https://openqa.opensuse.org/tests/2331276

This seems to have worked. I also prepared an MR to enable it on osd via /etc/openqa/openqa.ini by adding enable_force_result=true to [hook]job_done_hook_{incomplete,failed}.

#17 Updated by okurz about 2 months ago

merged https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/685 . As discussed please announce on openqa@suse.de, in matrix chat and Slack as well as extend github.com/os-autoinst/scripts/tree/master/README.md

#18 Updated by cdywan about 2 months ago

okurz wrote:

merged https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/685 . As discussed please announce on openqa@suse.de, in matrix chat and Slack as well as extend github.com/os-autoinst/scripts/tree/master/README.md

https://github.com/os-autoinst/scripts/pull/158

#19 Updated by okurz about 2 months ago

  • Related to action #107923: qem-bot: Ignore not-ok openQA jobs for specific incident based on openQA job comment added

#21 Updated by cdywan about 2 months ago

  • Status changed from Feedback to Resolved

cdywan wrote:

okurz wrote:

merged https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/685 . As discussed please announce on openqa@suse.de, in matrix chat and Slack as well as extend github.com/os-autoinst/scripts/tree/master/README.md

https://github.com/os-autoinst/scripts/pull/158

Docs merged.

#22 Updated by okurz about 1 month ago

  • Due date deleted (2022-05-18)

Also available in: Atom PDF