action #176730
opencoordination #102915: [saga][epic] Automated classification of failures
QA (public) - coordination #94105: [epic] Use feedback from openqa-investigate to automatically inform on github pull requests, open tickets, weed out automatically failed tests
[openqa-investigate] Investigation for job clusters creating multiple comments and producing error messages in the logs
0%
Description
Observation¶
- https://openqa.suse.de/tests/16679523#comments parallel_failed 2 comments "Automatic investigation jobs for job 16679524:" 07:12 "Automatic investigation jobs for job 16679526:" 07:12
- https://openqa.suse.de/tests/16679524#comments 1 comment with "Automatic investigation jobs for job 16679524:" 07:13
- https://openqa.suse.de/tests/16679525#comments parallel_failed
- https://openqa.suse.de/tests/16679526#comments 1 comment with "Automatic investigation jobs for job 16679526:" 07:13
- https://openqa.suse.de/tests/16679527#comments parallel_failed
- https://openqa.suse.de/tests/16679528#comments parallel_failed
So 16679523 has both comments from 16679524 and 16679526. With the same investigate job ids, though, so confusing, but somehow ok.
But what I found in the journal is weird:
% journalctl -u openqa-gru.service --since yesterday
...
Feb 06 07:11:37 openqa openqa-gru[19675]: Postponing to investigate job 16679526: waiting until 1 pending parallel job(s) finished
Feb 06 07:11:37 openqa openqa-gru[19678]: Postponing to investigate job 16679524: waiting until 1 pending parallel job(s) finished
Feb 06 07:12:47 openqa openqa-gru[23805]: Skipping investigation of job 16679526: job cluster is already being investigated, see comment on job 16679523
Feb 06 07:12:48 openqa openqa-gru[23805]: openqa-cli (342 /opt/os-autoinst-scripts/_common): (openqa-cli api --header User-Agent: openqa-investigate (https://github.com/os-autoinst/scripts) --host https://openqa.suse.de --retries=3 -X DELETE jobs/16679526/comments/1842627) stderr: >>>403 Forbidden<<<
Feb 06 07:12:48 openqa openqa-gru[23805]: openqa-cli (342 /opt/os-autoinst-scripts/_common): (openqa-cli api --header User-Agent: openqa-investigate (https://github.com/os-autoinst/scripts) --host https://openqa.suse.de --retries=3 -X DELETE jobs/16679526/comments/1842627) rc: 1 >>>{"error":"Administrator level required","error_status":403}<<<
The funny thing is that comment id 1842627 is the one from job 16679523, not 16679526: https://openqa.suse.de/tests/16679523#comment-1842627
That's probably why it fails to delete it, although with a very misleading error message; it should probably rather say that this comment does not exist.
Possibly this only happens when more than one job of the cluster fails.
Updated by tinita 2 months ago
- Subject changed from Investigation for job clusters creating multiple comments and producing error messages in the logs to [openqa-investigate] Investigation for job clusters creating multiple comments and producing error messages in the logs
- Description updated (diff)