Project

General

Profile

action #116494

Updated by okurz 5 months ago

## Observation

We have a lot of failed minion jobs.
Example:
https://openqa.suse.de/minion/jobs?id=5268904
```
result:
error: "<strong>Failed to save yast2_kdump-yast2-kdump-no-restart-info-20220913.</strong><br><pre>Unable
to fetch from origin master: Fetching origin\nremote: \nremote: ========================================================================\nremote:
\nremote: Your account has been blocked.\nremote: \nremote: ========================================================================\nremote:
\nfatal: Could not read from remote repository.\n\nPlease make sure you have the
correct access rights\nand the repository exists.\nerror: could not fetch origin</pre>"
```
The user has no rights to push the needles to the git repo anymore.

There is already an SD ticket about it: https://sd.suse.com/servicedesk/customer/portal/1/SD-98249

## Suggestions
* DONE: ~~Fix blocked account~~
* Review failed minion jobs and remove the ones that are about this ticket
* Ensure that the number of failed minion jobs is again below the alerting threshold
* There are three alerts now as well due to "webui-old" and "webui-test" dashboards. As decided in the weekly meeting 2022-09-16 please delete them.
* Ensure that only one alert "web UI: Too many Minion job failures alert" remains
* Cross-check alert state

## Rollback steps
* Unpause alert(s) "web UI: Too many Minion job failures alert"

Back