action #115814
closedRecord soft-failure in test suite installer_extended for product modules not available in early phase
Description
Observation¶
Failing openQA jobs: https://openqa.suse.de/tests/overview?arch=&flavor=&machine=&test=installer_extended&modules=&module_re=&distri=sle&version=15-SP5&build=25.1&groupid=129#
The list of expected modules (alphabetically sorted) is:
PackageHub
sle-ha
sle-module-NVIDIA-compute
sle-module-basesystem
sle-module-certifications
sle-module-containers
sle-module-desktop-applications
sle-module-development-tools
sle-module-legacy
sle-module-live-patching
sle-module-public-cloud
sle-module-python3
sle-module-server-applications
sle-module-transactional-server
sle-module-web-scripting
sle-we
The missing modules are PackageHub, sle-module-NVIDIA-compute and sle-module-certifications.
In the same way that with other task, we should try to avoid failures of this kind where the result cannot be predicted easily, in the sense that we don't know when this package will be available, could be in beta or could be a bit later. For that instead of having failure we will marking the job with soft-failure and then with very few failures we will be able to pay more attention to soft-failures.
Acceptance criteria¶
AC1: File a bug for this if doesn't exist
AC2: Improve existing die adding list of product module mismatching
AC3: Record soft-failure instead of using die when the product module list mismatching is the subset identified above.
Additional info¶
User function arrays_subset in utils.pm to actually give an error message with the mismatch.
Use the very same function to decide if die or soft-fail depending on the list of modules mismatching.