QA

Motivation¶

https://github.com/openSUSE/qem-dashboard/ on pages like http://dashboard.qam.suse.de/blocked shows which updates are blocked but to find more details what decision qem-bot does and why certain incidents are still blocked it can be multiple clicks and digging through log files of qem-bot to find out the current status in details, in particular when using @review:acceptable_for:incident_X labels on openQA jobs which still represent as simply red boxes on the dashboard. For making the life of reviewers easier the dashboard should be able to provide more details, at best on all levels accordingly.

Additional explanation by mgrifalconi:

I fear you see something only the bot logs https://gitlab.suse.de/qa-maintenance/bot-ng/-/jobs for approve incident jobs
You should see something like
"Ignoring failed job %s for incident %s due to openQA comment"
https://github.com/openSUSE/qem-bot/blob/463d5de225767791d0b6815435a0b341ce319c67/openqabot/approver.py#L261
BUT only if there is no other genuine failure "found" by the bot first. In that case, the bot will give up at first non-ignored failure and won't look at other acceptable ones.
Or you can see that it goes out of the queue and gets approved eventually :smile:
Would be nice to reflect that on the dashboard though, since at first glance it looks like something was not reviewed when instead it was.

Ideas¶

I am thinking of two things to do:

1. Provide the last relevant log lines referring to each incident on the page for each incident of the dashboard
2. use another pattern and color for the "acceptable_for" result. I am thinking of a yellow-green dashed pattern in the background of each result box

Acceptance criteria¶

• AC1: The reason why incidents are approvable / not-approvable are visible in http://dashboard.qam.suse.de/incident/$id
• AC2: http://dashboard.qam.suse.de/blocked visualizes by pattern+color when "acceptable_for" labels are in effect for certain results

Suggestions¶

• Consider forwarding the individual results as necessary from qem-bot to qem-dashboard, either as flags or complete log messages
• In qem-dashboard show the relevant data on /incident/$id
• In qem-dashboard show the effect of "approvable_for" on /blocked where applicable by pattern+color, e.g. yellow-green dashed pattern in the background of each result box where "approvable_for" is effective
• In qem-dashboard consider automatic cleanup of outdated details in the qem-dashboard database
• https://github.com/openSUSE/qem-bot/blob/463d5de225767791d0b6815435a0b341ce319c67/openqabot/approver.py#L129 is where the "acceptable_for" comment in qem-bot is parsed and evaluated. The challenge is that this is called during the "approval" command which is not syncing anything to the dashboard (yet).
• See also https://suse.slack.com/archives/C02DQJKULE4/p1722327940699429?thread_ts=1722235479.858789&cid=C02DQJKULE4):