Project

General

Profile

action #111506

qa-tools: qem-bot - Development results leaked to dashboard size:M

Added by osukup about 1 month ago. Updated 18 days ago.

Status:
Resolved
Priority:
High
Assignee:
Target version:
Start date:
2022-05-24
Due date:
2022-06-23
% Done:

0%

Estimated time:

Description

Observation

Development group 'pstivanin-security@Server-DVD-Updates' results synced to qem dashboard

We already filter out development jobs in https://github.com/openSUSE/qem-bot/blob/master/openqabot/aggrsync.py#L53-L68 in case of aggregates or https://github.com/openSUSE/qem-bot/blob/master/openqabot/incsyncres.py#L48-L61 but today pstivanin-security leaked to qem-dashboard in http://dashboard.qam.suse.de/incident/24199 when it shouldn't show it.

Acceptance criteria

  • AC1: Development jobs should not be part of incident approval
  • AC2: Dashboard no longer shows development jobs

Suggestions

Further notes

  • A "development job" is a job in a job group or in a parent job group where the name contains "Development".

History

#1 Updated by mgrifalconi about 1 month ago

After seeing the code used to filter out development job groups I have a question:

Isn't possible to get a list of al job groups in the "development" parent group, instead of filtering for job group name keywords?
To my understanding, developers know that the parent group is meant for development and not that some keywords will do the trick, but I might be the only one not knowing it.

#2 Updated by okurz about 1 month ago

  • Tags set to reactive work
  • Priority changed from Normal to High
  • Target version set to Ready

Well, it was started with just having job groups with "development" in the name and at that time parent groups didn't even exist yet.

But, yes, looking into the parent group as an alternative sounds like the way to go.

#3 Updated by mkittler about 1 month ago

  • Subject changed from qa-tools: qem-bot - Development results leaked to dashboard to qa-tools: qem-bot - Development results leaked to dashboard size:M
  • Description updated (diff)
  • Status changed from New to Workable

#4 Updated by okurz about 1 month ago

  • Description updated (diff)
  • Target version deleted (Ready)

#5 Updated by okurz about 1 month ago

  • Target version set to Ready

#6 Updated by osukup about 1 month ago

  • Status changed from Workable to In Progress
  • Assignee set to osukup

#7 Updated by osukup about 1 month ago

unfortunately openqa api call jobs/id returns only group and group_id but nothing about the parent group, so amount of API calls to openQA within bot is slightly higher now

#8 Updated by cdywan about 1 month ago

okurz wrote:

Well, it was started with just having job groups with "development" in the name

The code seems to recognize groups with names containing Devel, Test and Timo as well as specifically Development.

#9 Updated by osukup about 1 month ago

  • Status changed from In Progress to Feedback

cdywan wrote:

okurz wrote:

Well, it was started with just having job groups with "development" in the name

The code seems to recognize groups with names containing Devel, Test and Timo as well as specifically Development.

now it also checks the parent group ..., PR merged

#10 Updated by cdywan about 1 month ago

mgrifalconi wrote:

Isn't possible to get a list of al job groups in the "development" parent group, instead of filtering for job group name keywords?
To my understanding, developers know that the parent group is meant for development and not that some keywords will do the trick, but I might be the only one not knowing it.

I provided another PR to drop the keyword-based filtering since that's how I read your comment but I guess it's not clear what the expected workflow is.

With Ondřej's change we have both, the parent group as well as the filtering based on keywords in the name of the job group.

#11 Updated by mgrifalconi 23 days ago

Hello, I think it is happening again. With "Leap 15.4" and "Maintenance: Leap 15.3". Please see https://suse.slack.com/archives/C02D16TCP99/p1654676089877079 and also http://dashboard.qam.suse.de/blocked by searching for "Leap" in the page.

#12 Updated by cdywan 23 days ago

mgrifalconi wrote:

Hello, I think it is happening again. With "Leap 15.4" and "Maintenance: Leap 15.3". Please see https://suse.slack.com/archives/C02D16TCP99/p1654676089877079 and also http://dashboard.qam.suse.de/blocked by searching for "Leap" in the page.

https://github.com/openSUSE/qem-bot/pull/34 is the refactor claiming to handle parent groups as well. It didn't add any unit test coverage so I suggest we get this unit-tested first and identify the fix that way.

#13 Updated by osukup 23 days ago

  • Status changed from Feedback to In Progress

found problem in filter code, "name" keys has different meaning ... so using magic number is only way

#14 Updated by okurz 23 days ago

osukup wrote:

found problem in filter code, "name" keys has different meaning ... so using magic number is only way

Better make it a "configuration value". It's only a "magic number" if there is no proper explanation to what it means :) And, no, a comment in source code does not count :)

#15 Updated by openqa_review 23 days ago

  • Due date set to 2022-06-23

Setting due date based on mean cycle time of SUSE QE Tools

#16 Updated by osukup 21 days ago

okurz wrote:

osukup wrote:

found problem in filter code, "name" keys has different meaning ... so using magic number is only way

Better make it a "configuration value". It's only a "magic number" if there is no proper explanation to what it means :) And, no, a comment in source code does not count :)

now is as one initialized variable in openqabot/__init__.py

#17 Updated by osukup 21 days ago

  • Status changed from In Progress to Feedback

merged

#18 Updated by osukup 18 days ago

  • Status changed from Feedback to Resolved

should filter out develepment result correctly now

#19 Updated by cdywan 18 days ago

osukup wrote:

should filter out develepment result correctly now

For the record: https://github.com/openSUSE/qem-bot/pull/40

Also available in: Atom PDF