QA » openQA Project » openQA Infrastructure

SR to add a retry: https://gitlab.suse.de/openqa/monitor-o3/-/merge_requests/4

Note that there's another flaw (which I haven't changed so far): If o3 is down we will needlessly power cycle all (physical) worker hosts.

I've merged the SR and also a 2nd one to avoid the problem mentioned in the previous comment.

I've tested the different cases locally (using power status or invalid IPMI commands instead of power cycle) so it should work.

I've also just retriggered a pipeline but it cannot find the script anymore. So I'll have a look what's wrong.

The good case is now working with https://gitlab.suse.de/openqa/monitor-o3/-/commit/1b659e6cd2d3f11a352b2712892ac54a385be38b: https://gitlab.suse.de/openqa/monitor-o3/-/pipelines/350706

mkittler wrote:

The good case is now working with https://gitlab.suse.de/openqa/monitor-o3/-/commit/1b659e6cd2d3f11a352b2712892ac54a385be38b: https://gitlab.suse.de/openqa/monitor-o3/-/pipelines/350706

I don't think so...

https://gitlab.suse.de/openqa/monitor-o3/-/pipelines/350704

/scripts-5544-902568/step_script: line 157: ./monitor-and-recover: No such file or directory

But since then @okurz added the mentioned commit and the re-triggered pipeline works: https://gitlab.suse.de/openqa/monitor-o3/-/pipelines/350706