AdamWill wrote:
The grey "commented" badge for builds is meant to indicate, I think, that at least someone has looked at all the failures: a build gets it if all failed jobs have a comment.
well, a "commented" badge basically only means that a comment is present. It would need to be checked in detail what the comment is.
The RETRY system kind of breaks this, because any retried job gets an automated comment from the retry system. So if all failed jobs for a build were auto-retried, the build will get the grey 'commented' badge even though no human has reviewed the failures.
There are multiple other systems which can generate a comment, e.g. we have github.com/os-autoinst/scripts/ which are triggered by http://open.qa/docs/#_enable_custom_hook_scripts_on_job_done_based_on_result so we have comments from automatic systems which are not humans. Then a comment without a bugref would mean "there is additional information, maybe from a human, maybe from a bot, but no known referenced issue yet"
Not sure what would be the best way to deal with this. Indicate retried jobs in some other way than a comment? Filter retried-job comments out from the badge calculation?
I see the following options:
- Adapting the processes around the refined definition that the comment icon just means "presence of a comment with potentially more information"
- Your suggestion that retry was applied but without a job comment. This could also be configurable.
- Come up with a new type to distinguish from plain "comments" and also not including in badge calculation. I would not just filter out "special comments" from the badge calculation as that would be potentially hard to understand why certain comments are included and others not. Better make it explicit by having a separate type.
I think option 3 is becoming too confusing and we should avoid that. I could live with option 2 but configurable or rethink and actually not have a comment in openQA jobs. The comment was introduced in https://github.com/os-autoinst/openQA/pull/4582/commits/4935386b6b279244088f1d1c049a93a55f50ca0d and it was not part of the original design. @mkittler WDYT about this, do we actually need the retry comment in openQA jobs? If we would not have a comment would we have openqa-investigate runs on jobs that we retry anyway?
Of course, option 1 would be easiest for us :)