Project

General

Profile

Actions

openqa-force-result #109857

closed

coordination #89062: [epic] Simplify review for SUSE QAM

Secure auto-review+force_result size:M auto_review:"Failed to download gobbledeegoop":force_result:softfailed

Added by okurz over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Feature requests
Target version:
Due date:

Description

Motivation

See #104923 . We should only allow a trusted set of people to influence test results on o3+osd using force_result in auto-review tickets.

  • AC1: Only a trusted, known set of people can influence test results on o3+osd using force_result in auto-review tickets

Suggestions

  • Define a new "tracker" in progress.opensuse.org (we already use "action" and "coordination", could add another one there, e.g. "openqa-force-result")
  • Configure "Non member"-permissions to disallow create+update of tickets.
  • Create new sub-project of openqatests using only that tracker (where we trust members, e.g. the "QA" project tree)
  • Extend openqa-label-known-issues to only look into that project for auto-review+force-result (optionally only the specific tracker)

This paired with a progress project where we trust members plus openqa-label-known-issues to only look at a certain tracker for "openqa-force-result" tickets would mean that only members of the specified project can create+update "openqa-force-result" tickets in the "openqa-force-result" tracker.

Rejected alternative: Create a private ticket but this has the significant drawback that soon openQA jobs will be labeled with tickets automatically and not everybody can see the tickets

Actions #1

Updated by okurz over 2 years ago

  • Parent task set to #104923
Actions #2

Updated by livdywan over 2 years ago

  • Subject changed from Secure auto-review+force_result to Secure auto-review+force_result size:M
  • Status changed from New to Workable
Actions #3

Updated by livdywan over 2 years ago

  • Status changed from Workable to In Progress
  • Assignee set to livdywan

I'll give this a go. Maybe as a side effect this will teach me some new Readmine foo

Actions #4

Updated by livdywan over 2 years ago

I created openQA investigation findings which uses the also newly created openqa-force-result and removed the ability for Non member to add or delete tickets or notes.

My user who isn't part of any project now receives the following message when trying to file a ticket: The project doesn't have any trackers for which you can create an issue.

Actions #5

Updated by openqa_review over 2 years ago

  • Due date set to 2022-05-18

Setting due date based on mean cycle time of SUSE QE Tools

Actions #7

Updated by okurz over 2 years ago

cdywan wrote:

I created openQA investigation findings which uses the also newly created openqa-force-result and removed the ability for Non member to add or delete tickets or notes.

My user who isn't part of any project now receives the following message when trying to file a ticket: The project doesn't have any trackers for which you can create an issue.

I consider https://progress.opensuse.org/projects/openqa-investigation-findings/ rather long and also I am afraid that it will mislead people to report tickets in there which should be in "openqatests" instead. How about "openqa-auto-review"?

Actions #8

Updated by livdywan over 2 years ago

okurz wrote:

cdywan wrote:

I created openQA investigation findings which uses the also newly created openqa-force-result and removed the ability for Non member to add or delete tickets or notes.

My user who isn't part of any project now receives the following message when trying to file a ticket: The project doesn't have any trackers for which you can create an issue.

I consider https://progress.opensuse.org/projects/openqa-investigation-findings/ rather long and also I am afraid that it will mislead people to report tickets in there which should be in "openqatests" instead. How about "openqa-auto-review"?

This is a short and readable URL in my book, but I really don't care so I'll rename it

Actions #9

Updated by livdywan over 2 years ago

  • Status changed from In Progress to Feedback

cdywan wrote:

okurz wrote:

cdywan wrote:

I created openQA investigation findings which uses the also newly created openqa-force-result and removed the ability for Non member to add or delete tickets or notes.

My user who isn't part of any project now receives the following message when trying to file a ticket: The project doesn't have any trackers for which you can create an issue.

I consider https://progress.opensuse.org/projects/openqa-investigation-findings/ rather long and also I am afraid that it will mislead people to report tickets in there which should be in "openqatests" instead. How about "openqa-auto-review"?

This is a short and readable URL in my book, but I really don't care so I'll rename it

openqa-auto-review it is.

And only honoring force result expressions with the new tracker would seem to work regardless of the project. So at this point we can use it or stick to projects we were already using.

Actions #10

Updated by okurz over 2 years ago

So what are your next steps planned? I suggest you actually test this

Actions #11

Updated by livdywan over 2 years ago

okurz wrote:

So what are your next steps planned? I suggest you actually test this

Please provide constructive comments where possible by e.g. responding to my point on not needing the sub-project or making concrete suggestions like "Have you tested this yet? How about moving a ticket to the new tracker and using force_result: with a test job".

Actions #12

Updated by livdywan over 2 years ago

  • Subject changed from Secure auto-review+force_result size:M to Secure auto-review+force_result size:M auto_review:"Failed to download gobbledeegoop":force_result:softfailed
openqa-clone-job --within-instance https://openqa.opensuse.org/t2330754 TEST=eggs RETRY=0 HDD_1=gobbledeegoop.qcow2 _GROUP=0
Actions #13

Updated by livdywan over 2 years ago

  • Tracker changed from action to openqa-force-result
  • Category deleted (Feature requests)
  • Start date deleted (2022-04-12)
Actions #14

Updated by livdywan over 2 years ago

Apparently this was matching "something" else before. And this also had to be re-enabled in the config on o3. Let's see if this will work: https://openqa.opensuse.org/tests/2331276

Actions #15

Updated by okurz over 2 years ago

  • Category set to Feature requests
Actions #16

Updated by livdywan over 2 years ago

cdywan wrote:

Apparently this was matching "something" else before. And this also had to be re-enabled in the config on o3. Let's see if this will work: https://openqa.opensuse.org/tests/2331276

This seems to have worked. I also prepared an MR to enable it on osd via /etc/openqa/openqa.ini by adding enable_force_result=true to [hook]job_done_hook_{incomplete,failed}.

Actions #17

Updated by okurz over 2 years ago

merged https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/685 . As discussed please announce on openqa@suse.de, in matrix chat and Slack as well as extend github.com/os-autoinst/scripts/tree/master/README.md

Actions #18

Updated by livdywan over 2 years ago

okurz wrote:

merged https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/685 . As discussed please announce on openqa@suse.de, in matrix chat and Slack as well as extend github.com/os-autoinst/scripts/tree/master/README.md

https://github.com/os-autoinst/scripts/pull/158

Actions #21

Updated by livdywan over 2 years ago

  • Status changed from Feedback to Resolved

cdywan wrote:

okurz wrote:

merged https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/685 . As discussed please announce on openqa@suse.de, in matrix chat and Slack as well as extend github.com/os-autoinst/scripts/tree/master/README.md

https://github.com/os-autoinst/scripts/pull/158

Docs merged.

Actions #22

Updated by okurz over 2 years ago

  • Due date deleted (2022-05-18)
Actions

Also available in: Atom PDF