Project

General

Profile

action #81106

coordination #69310: [epic] SUSE QA tools team ticket process helpers

test out chat service notifications, e.g. matrix, from github actions size:M

Added by okurz 12 months ago. Updated 6 days ago.

Status:
Feedback
Priority:
Low
Assignee:
-
Target version:
Start date:
2020-12-16
Due date:
% Done:

0%

Estimated time:

Description

Motivation

Based on #69310 we find useful limits, SLOs, queries to check but we are unsure about notifications, e.g. how would we react to GHA pipeline failures. Maybe it helps to receive slack notifications

Acceptance criteria

  • AC1: GHA pipeline failures in os-autoinst/scripts trigger matrix notifications
  • AC2: team members are aware of where the notifications come from and what needs to be done
  • AC3: We are not annoyed by too many frickin' unactionable alerts

Suggestions

For slack one can
try out
Slack Messaging
in https://github.com/os-autoinst/scripts GHA pipelines

Don't spam channels but definitely not community channels :)


Related issues

Related to openQA Infrastructure - action #77317: chat bot to conduct daily checks, alerts, reminders, etc.New2020-11-10

Related to QA - action #102059: Integrate the Slack feed notifications feature for progress queriesResolved2021-11-082021-11-19

History

#1 Updated by okurz 12 months ago

  • Tracker changed from coordination to action

#2 Updated by okurz 12 months ago

  • Project changed from openQA Project to QA
  • Category deleted (Feature requests)

#3 Updated by okurz 12 months ago

  • Related to action #77317: chat bot to conduct daily checks, alerts, reminders, etc. added

#4 Updated by okurz 8 months ago

  • Priority changed from Normal to Low

#5 Updated by okurz 7 months ago

  • Target version changed from Ready to future

this was an idea by the team and for the team but apparently there is not much interest so I will move it out of the backlog for now

#6 Updated by okurz 4 months ago

  • Subject changed from test out rocket chat notifications from github actions to test out chat service notifications, e.g. rocket chat, from github actions

#7 Updated by cdywan 3 months ago

  • Subject changed from test out chat service notifications, e.g. rocket chat, from github actions to test out chat service notifications, e.g. slack, from github actions
  • Description updated (diff)

#8 Updated by okurz 3 months ago

  • Subject changed from test out chat service notifications, e.g. slack, from github actions to test out chat service notifications, e.g. matrix, from github actions
  • Description updated (diff)
  • Target version changed from future to Ready

I am not really motivated to support proprietary tools before free software so I suggest to use matrix/element first

#9 Updated by VANASTASIADIS 3 months ago

  • Assignee set to VANASTASIADIS

#10 Updated by VANASTASIADIS 3 months ago

  • Status changed from Workable to In Progress

#11 Updated by VANASTASIADIS 3 months ago

There is a working matrix solution (currently on a personal test repo, but easily reproducible in any workflow with minor additions). I am wondering however:

  • currently in the scripts repo there are 2 jobs, containing 3 steps in the workflow: JOB 1: a) check WIP limits b) set due dates JOB 2: c) run ci tests on push

I'm wondering: since the first job is a scheduled job, should a notification be sent in every failure? That's certainly easier (and less complex), but if for example the job fails on a Friday evening, by monday we'd have a lot of spam in the chat.

Another solution would be saving the previous condition in a file in the repo, and comparing to see if anything changed. That would lead to notifications only the first time something fails. But it would still miss other cases: for example, a case where the same job/step fails but for a different reason.

I think it's simpler in the case of https://github.com/os-autoinst/qa-tools-backlog-assistant: notify only on status change for every query. If "overall" backlog is off limits, you get one notification: the next one will be when it's inside limits again.

Depending on the job in question, different ways of notifying may be prefferable: for some jobs only once on every change, on others only one report in fixed intervals... I would appreciate other opinions and thoughts here.

#12 Updated by okurz 3 months ago

  • Subject changed from test out chat service notifications, e.g. matrix, from github actions to test out chat service notifications, e.g. matrix, from github actions size:M
  • Description updated (diff)

#13 Updated by cdywan 3 months ago

VANASTASIADIS wrote:

There is a working matrix solution (currently on a personal test repo, but easily reproducible in any workflow with minor additions). I am wondering however:

What solution is that? Do you have a proof of concept implementing this?

  • currently in the scripts repo there are 2 jobs, containing 3 steps in the workflow: JOB 1: a) check WIP limits b) set due dates JOB 2: c) run ci tests on push

I'm wondering: since the first job is a scheduled job, should a notification be sent in every failure? That's certainly easier (and less complex), but if for example the job fails on a Friday evening, by monday we'd have a lot of spam in the chat.

What do you consider "a lot"? I would suggest to aim for one notification a day. I don't care if we see the exact same the day after, that just means we need to catch up.

#15 Updated by VANASTASIADIS 2 months ago

  • Status changed from In Progress to Feedback

#16 Updated by cdywan 2 months ago

We still need these items here:

  • We need #suse-qe-tools to receive notifications
  • Add secrets to the GitHub pipeline
  • We need a bot account for Matrix

#17 Updated by okurz 27 days ago

  • Related to action #102059: Integrate the Slack feed notifications feature for progress queries added

#18 Updated by okurz 21 days ago

@VANASTASIADIS what are your own plans on this? Should we unassign you and pick it up within SUSE QE Tools?

#19 Updated by VANASTASIADIS 6 days ago

  • Assignee deleted (VANASTASIADIS)

okurz So, for matrix this should be working, as long as someone adds MATRIX_ACCESS_TOKEN and MATRIX_ROOM_ID to the secrets. I don't have the permissions to add secrets, so someone with the appropriate permissions should do that.

In addition, I see that this has rolled back to being slack-centric. So I will unassign and feel free to assign and add the secrets, or proceed with a slack implementation. If it's not too urgent and you have no free hands, I can tackle the slack implementation too, after I'm done with a couple of qe-core tickets. Ping me in that case.

Also available in: Atom PDF