action #157204
closedcoordination #99303: [saga][epic] Future improvements for SUSE Maintenance QA workflows with fully automated testing, approval and release
coordination #155671: [epic] Better handling of SLE maintenance test review
Sync openQA job removal events to qem-dashboard listening to AMQP events size:M
0%
Description
Motivation¶
https://suse.slack.com/archives/C02CLB8TZP1/p1709892527534149?thread_ts=1709883106.021479&cid=C02CLB8TZP1
When openQA jobs are deleted then the according reference in qem-dashboard should also be removed. Listen to AMQP events to sync the removal accordingly
Acceptance criteria¶
- AC1: SLE maintenance openQA jobs previously blocking SLE maintenance updates on http://dashboard.qam.suse.de/blocked do not block approval after such openQA jobs are deleted from the openQA database
Suggestions¶
- Use TDD: Extend https://github.com/openSUSE/qem-dashboard/blob/main/t/amqp.t and ensure there is a failing test first
- Extend https://github.com/openSUSE/qem-dashboard/blob/08cea810f936faeb6af35b645270d85f6569c6b9/lib/Dashboard/Model/AMQP.pm#L33 to update the database entry accordingly or delete, whatever is applicable
- For all current openQA job result entries in the dashboard database crosscheck if there are entries for jobs that do not exist anymore in the openQA database. Remove accordingly.
- Verify operation in production: E.g. create an artificial, failed openQA job in OSD for a non-critical SLE maintenance update, wait till it shows up as blocking on http://dashboard.qam.suse.de/blocked or in log files of the qem-bot "approve" cycle, remove the job over
openqa-cli -X delete jobs/$idagain and verify that http://dashboard.qam.suse.de/blocked does not show up as blocked on that job anymore
Out of scope¶
- Regular cleanup of results when we missed or have otherwise not received according AMQP events
Updated by jbaier_cz 9 months ago
https://github.com/openSUSE/qem-dashboard/pull/1017 should be enough to listen for delete events and act accordingly.
Updated by jbaier_cz 9 months ago
- Status changed from Feedback to Resolved
After creating (and cancelling) a few incident jobs, I can see that every individual job delete will create a corresponding line in the server log (for the dasboard-amqp-watcher.service) and a decrease of job count on the dashboard for that incident. That should solve the qem-bot blocking for deleted openQA jobs.
Apr 12 18:26:06 qam2 dashboard[3026]: [3026] [i] delete 14017258
Updated by okurz 6 months ago
- Related to action #164733: qem-dashboard (and hence qem-bot) see a job as failed even though it's marked as softfailed since > 30 days in openQA size:M added