openqa-force-result #109857
closedQA - coordination #91646: [saga][epic] SUSE Maintenance QA workflows with fully automated testing, approval and release
coordination #89062: [epic] Simplify review for SUSE QAM
Secure auto-review+force_result size:M auto_review:"Failed to download gobbledeegoop":force_result:softfailed
Description
Motivation¶
See #104923 . We should only allow a trusted set of people to influence test results on o3+osd using force_result in auto-review tickets.
- AC1: Only a trusted, known set of people can influence test results on o3+osd using force_result in auto-review tickets
Suggestions¶
- Define a new "tracker" in progress.opensuse.org (we already use "action" and "coordination", could add another one there, e.g. "openqa-force-result")
- Configure "Non member"-permissions to disallow create+update of tickets.
- Create new sub-project of openqatests using only that tracker (where we trust members, e.g. the "QA" project tree)
- Extend openqa-label-known-issues to only look into that project for auto-review+force-result (optionally only the specific tracker)
This paired with a progress project where we trust members plus openqa-label-known-issues to only look at a certain tracker for "openqa-force-result" tickets would mean that only members of the specified project can create+update "openqa-force-result" tickets in the "openqa-force-result" tracker.
Rejected alternative: Create a private ticket but this has the significant drawback that soon openQA jobs will be labeled with tickets automatically and not everybody can see the tickets
Updated by livdywan about 2 years ago
- Subject changed from Secure auto-review+force_result to Secure auto-review+force_result size:M
- Status changed from New to Workable
Updated by livdywan about 2 years ago
- Status changed from Workable to In Progress
- Assignee set to livdywan
I'll give this a go. Maybe as a side effect this will teach me some new Readmine foo
Updated by livdywan about 2 years ago
I created openQA investigation findings which uses the also newly created openqa-force-result and removed the ability for Non member to add or delete tickets or notes.
My user who isn't part of any project now receives the following message when trying to file a ticket: The project doesn't have any trackers for which you can create an issue.
Updated by openqa_review about 2 years ago
- Due date set to 2022-05-18
Setting due date based on mean cycle time of SUSE QE Tools
Updated by livdywan about 2 years ago
Updated by okurz about 2 years ago
cdywan wrote:
I created openQA investigation findings which uses the also newly created openqa-force-result and removed the ability for Non member to add or delete tickets or notes.
My user who isn't part of any project now receives the following message when trying to file a ticket:
The project doesn't have any trackers for which you can create an issue.
I consider https://progress.opensuse.org/projects/openqa-investigation-findings/ rather long and also I am afraid that it will mislead people to report tickets in there which should be in "openqatests" instead. How about "openqa-auto-review"?
Updated by livdywan about 2 years ago
okurz wrote:
cdywan wrote:
I created openQA investigation findings which uses the also newly created openqa-force-result and removed the ability for Non member to add or delete tickets or notes.
My user who isn't part of any project now receives the following message when trying to file a ticket:
The project doesn't have any trackers for which you can create an issue.I consider https://progress.opensuse.org/projects/openqa-investigation-findings/ rather long and also I am afraid that it will mislead people to report tickets in there which should be in "openqatests" instead. How about "openqa-auto-review"?
This is a short and readable URL in my book, but I really don't care so I'll rename it
Updated by livdywan about 2 years ago
- Status changed from In Progress to Feedback
cdywan wrote:
okurz wrote:
cdywan wrote:
I created openQA investigation findings which uses the also newly created openqa-force-result and removed the ability for Non member to add or delete tickets or notes.
My user who isn't part of any project now receives the following message when trying to file a ticket:
The project doesn't have any trackers for which you can create an issue.I consider https://progress.opensuse.org/projects/openqa-investigation-findings/ rather long and also I am afraid that it will mislead people to report tickets in there which should be in "openqatests" instead. How about "openqa-auto-review"?
This is a short and readable URL in my book, but I really don't care so I'll rename it
openqa-auto-review it is.
And only honoring force result expressions with the new tracker would seem to work regardless of the project. So at this point we can use it or stick to projects we were already using.
Updated by okurz almost 2 years ago
So what are your next steps planned? I suggest you actually test this
Updated by livdywan almost 2 years ago
okurz wrote:
So what are your next steps planned? I suggest you actually test this
Please provide constructive comments where possible by e.g. responding to my point on not needing the sub-project or making concrete suggestions like "Have you tested this yet? How about moving a ticket to the new tracker and using force_result: with a test job".
Updated by livdywan almost 2 years ago
- Subject changed from Secure auto-review+force_result size:M to Secure auto-review+force_result size:M auto_review:"Failed to download gobbledeegoop":force_result:softfailed
openqa-clone-job --within-instance https://openqa.opensuse.org/t2330754 TEST=eggs RETRY=0 HDD_1=gobbledeegoop.qcow2 _GROUP=0
Updated by livdywan almost 2 years ago
- Tracker changed from action to openqa-force-result
- Category deleted (
Feature requests) - Start date deleted (
2022-04-12)
Updated by livdywan almost 2 years ago
Apparently this was matching "something" else before. And this also had to be re-enabled in the config on o3. Let's see if this will work: https://openqa.opensuse.org/tests/2331276
Updated by okurz almost 2 years ago
- Category set to Feature requests
https://github.com/os-autoinst/scripts/pull/155 as a related improvement
Updated by livdywan almost 2 years ago
cdywan wrote:
Apparently this was matching "something" else before. And this also had to be re-enabled in the config on o3. Let's see if this will work: https://openqa.opensuse.org/tests/2331276
This seems to have worked. I also prepared an MR to enable it on osd via /etc/openqa/openqa.ini by adding enable_force_result=true to [hook]job_done_hook_{incomplete,failed}.
Updated by okurz almost 2 years ago
merged https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/685 . As discussed please announce on openqa@suse.de, in matrix chat and Slack as well as extend github.com/os-autoinst/scripts/tree/master/README.md
Updated by livdywan almost 2 years ago
okurz wrote:
merged https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/685 . As discussed please announce on openqa@suse.de, in matrix chat and Slack as well as extend github.com/os-autoinst/scripts/tree/master/README.md
Updated by okurz almost 2 years ago
- Related to action #107923: qem-bot: Ignore not-ok openQA jobs for specific incident based on openQA job comment size:M added
Updated by livdywan almost 2 years ago
- Status changed from Feedback to Resolved
cdywan wrote:
okurz wrote:
merged https://gitlab.suse.de/openqa/salt-states-openqa/-/merge_requests/685 . As discussed please announce on openqa@suse.de, in matrix chat and Slack as well as extend github.com/os-autoinst/scripts/tree/master/README.md
Docs merged.