Actions
action #119479
openopenqABot pipeline failed after runner getting stuck for 1h0m0s size:M
Start date:
2022-10-21
Due date:
% Done:
0%
Estimated time:
Description
Motivation¶
The openqABot pipeline failed. There's no logs and it seems like the bot never returns: https://gitlab.suse.de/qa-maintenance/openQABot/-/jobs/1212440, https://gitlab.suse.de/qa-maintenance/openQABot/-/jobs/1147725 and https://gitlab.suse.de/qa-maintenance/openQABot/-/jobs/1147636 (which aren't in the maintenance window)
++ ./oqabot.py --verbose --only-mr --metadata /etc/openqabot/mr.yml
41++ tee openqabot_0.log
43ERROR: Job failed: execution took longer than 1h0m0s seconds
A suspicion is that the GitLab runner never returned since otherwise we would see an error or log output. This occurred during the maintenance window.
Acceptance criteria¶
- AC1: It is ensured that gitlab CI jobs do not time out with no helpful information
Suggestions¶
- Raise to SUSE-IT, ask about the gitlab instance and runners and ask to review logs at the time of the failure
- Increase the timeout e.g. Settings -> CI / CD -> General pipelines -> Timeout
- Look at job history for other failed jobs to find a pattern https://gitlab.suse.de/qa-maintenance/openQABot/-/jobs?statuses=FAILED
Updated by jbaier_cz over 1 year ago
Also please note, we have
artifacts:
when: always
in the pipeline, so we should have the log uploaded even if the pipeline itself fail (so this is also an indication of a runner failure than a job failure).
Updated by livdywan over 1 year ago
- Subject changed from openqABot pipeline failed after runner getting stuck for 1h0m0s to openqABot pipeline failed after runner getting stuck for 1h0m0s size:M
- Description updated (diff)
- Status changed from New to Workable
Updated by livdywan 9 months ago
- Related to action #133457: salt-states-openqa gitlab CI pipeline aborted with error after 2h of execution size:M added
Actions