Project

General

Profile

action #58700

Updated by okurz over 1 year ago

## Observation

See the worker log from every job. The message "Test schedule has changed, reloading test_order.json" is repeated very often when we would expect to

## Acceptance criteria

* **AC1:** The test order is only re-evaluated if necessary
* **AC2:** Any logging output corresponds to actual actions, i.e. message "Test schedule has changed, reloading test_order.json" when test schedule has actually changed, e.g. worker log from https://openqa.suse.de/tests/latest?arch=x86_64&distri=sle&flavor=Server-DVD-Incidents-Kernel&machine=64bit&test=ltp_pty&version=15 only shows a single (or limited) "Test schedule has changed…"
* **AC3:** the test order is not re-evaluated for static schedules, e.g. worker log from https://openqa.opensuse.org/tests/latest?arch=x86_64&distri=opensuse&flavor=DVD&machine=64bit&test=textmode&version=Tumbleweed does *not* show "Test schedule has changed, reloading test_order.json"

## Suggestions

* Research what was the original idea of #53948 and https://github.com/os-autoinst/openQA/pull/2302 together with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/8329 and the approaches of https://github.com/os-autoinst/openQA/pull/2439 and https://github.com/os-autoinst/openQA/pull/2445 trying to fix the problem
* Find a good test case based on the openQA scenarios mentioned above
* Ensure there is sufficient unit test coverage within os-autoinst for the feature
* Fix the real problem :)
job

Back