action #80908

Updated by okurz 10 months ago

## Motivation

We want to upgrade more often but not disrupt openQA jobs on package upgrades as well as re-read configuration whenever a job finishes

## Acceptance criteria
* **AC1:** openQA worker packages can be upgraded continously without interrupting currently running openQA jobs
* **AC2:** openQA workers read updated configuration, e.g. WORKER_CLASS, whenever they are ready to pick up new jobs

## Further details

One could try what apache does with `apache2ctl graceful` or `systemctl reload apache2`, e.g. see

The restart of openQA workers could be simply prevented or delayed, e.g. with `SendSIGKILL=` in the openQA worker systemd service definitions which every openQA user is free to do, but then we could potentially wait hours until the service restarts if ever. Maybe we can still add a "graceful-stop" mode, wait a useful time for all jobs to finish and then restart (or even reboot the host).