notifications about failed and unreviewed jobs
As a test reviewer I want to get informed about new failed and unreviewed jobs so that I can find
- AC1: A failed job with no bugref or label is sent as an individual notification, e.g. using the AMQP IRC bridge in [#opensuse-factory](irc://chat.freenode.net/opensuse-factory) or [#openqa-test](irc://irc.suse.de/openqa-test)
- AC2: A failed job with automatically carried over bugref triggers no such notification
- AC3: The notification references the label content (, e.g. "bsc#1234" as link)
provide logic that can find "failed unreviewed jobs", e.g. subscribe to job events, look for failed jobs, query openQA api for this job if there is any comment on the job, if not, publish a new event to be picked up by the amqp-irc bridge. This can be done within openQA or in a supporting infrastructure. Probably helpful as an individual event on AMQP.
#5 Updated by mkittler over 1 year ago
- Status changed from New to In Progress
- Assignee set to mkittler
- Target version changed from Ready to Current Sprint
To compute the flag I need to find the bug reference anyways so I've just put the bug reference itself in the
job.done event. If it is absent (set to
null) one can consider the job unreviewed. The result (e.g. failed) is already part of the notification. So you basically have the information you need to figure that there's a failed and unreviewed job. Carried over bug references are considered as well (for AC2).
okurz Not sure what you mean with "as a link" in AC3 since this is just an AMQP message. But to ease creation of a link on the consumer-side I've also added the resolved bug URL (in case a bug reference is present).
#6 Updated by okurz over 1 year ago
Could work. https://github.com/okurz/openqa_irc/blob/master/Bot/BasicBot/Pluggable/Module/OpenQA.pm is an examplatory place where one can integrate the transformation into "human-digestable" format. Maybe you want to try that?
#8 Updated by mkittler over 1 year ago
Some of the dependencies of that IRC bot are missing (not in
devel:languages:perl and some not even in
devel:languages:perl:CAPN:*). I tried to install missing Perl modules via
cpan but that failed with a permission error trying to install something under
/usr/bin. I don't want to mess my system for this so I didn't proceed.
Maybe it is better to add this to https://github.com/openSUSE/suse_msg anyways? That one I had successfully running on my system before and all its dependencies are packaged.
#9 Updated by mkittler over 1 year ago
So far we have:
[11:17] [Notice] -hermes an #openqa-events- openQA job finished with result failed: https://openqa.suse.de/t2390060
I could change that to:
[11:17] [Notice] -hermes an #openqa-events- openQA job finished with result failed (review outstanding): https://openqa.suse.de/t2390060
Then someone interested in those notifications could simply configure the IRC client to produce a notification for messages containing
failed (review outstanding). That should be possible with Konversion. Not sure whether it is a common feature, though.
#16 Updated by mkittler over 1 year ago
Apparently it doesn't work in production. E.g. on
#openqa-events we get
[13:19] [Notice] -hermes an #openqa-events- openQA job finished with result failed (None): https://openqa.suse.de/t2425829. The brace should contain the bugref or 'review pending'. I have actually tested it locally (see PR).
dheidler Have you deployed the changes for the Python bot yet (and restarted the service)?
#17 Updated by mkittler over 1 year ago
Seems like the
None is emitted when there's no bugref:
[12:11] [Notice] -hermes an #openqa-events- openQA job finished with result failed (None): https://openqa.suse.de/t2430595 [12:11] [Notice] -hermes an #openqa-events- openQA job restarted: https://openqa.suse.de/t2430831 [12:11] [Notice] -hermes an #openqa-events- openQA job finished with result passed: https://openqa.suse.de/t2430997 [12:12] [Notice] -hermes an #openqa-events- openQA job finished with result failed (bsc#1073390): https://openqa.suse.de/t2429267
So likely just a mistake on the Python-side where my change is obviously already deployed.
#18 Updated by mkittler over 1 year ago
- Status changed from In Progress to Resolved
Works now after fixing a slight mistake on the Python-side, e.g.:
[16:08] [Notice] -hermes an #openqa-events- openQA job finished with result failed (bsc#1106085): https://openqa.suse.de/t2439840 [16:10] [Notice] -hermes an #openqa-events- openQA job finished with result failed (review pending): https://openqa.suse.de/t2440097
#19 Updated by okurz over 1 year ago
I appreciate the work you did but I doubt we are there to have the user story fulfilled. If you prefer we can continue in another ticket but have to be senseful about what users of openQA ask for: Same as in #16276#note-15, Are you sure AC2 is covered? How about we talk in person next week in office when you address me?
#22 Updated by mkittler over 1 year ago
I have just added a flag to tell whether the bug was carried-over. But when writing the test for it I've noticed that this is actually not that useful. I would have to construct the rare (test) case where a bugref is present at the time the job fails but it hasn't been added via take over but instead by a user while the test was still running.
And thinking further, you actually don't even need to distinguish the carry over. If there's no bugref at the time the job failed you will get "review pending" on IRC. That implies that no bugref has been carried over. Otherwise you wouldn't get "review pending" and the carried over bugref instead. So just setup your IRC client to notify you if a message contains "review pending" and you're good. So AC2 is fulfilled, indeed.
Since I have implemented it anyways, we can still add an explicit "carried over flag" to the AMQP message if you like. Just the test for the negative case would be missing.