action #105124
closed[qe-core] qam-incidentinstall is green even when installation is skipped
0%
Description
Hello,
tt seems that openQA treats as valid result (i.e. green) an empty installation test.
For example, looking at qam-incidentinstall of this MU https://openqa.suse.de/tests/overview?distri=sle&version=15-SP3&build=%3A22399%3Arpmlint&groupid=367, we can see that it's green for all 4 architectures. Even the generated report looks good:
sled_15-SP3_x86_64 => passed: http://qam.suse.de/testreports/SUSE:Maintenance:22399:262470/install_logs/sled_15-SP3_x86_64.log
sles_15-SP3_aarch64 => passed: http://qam.suse.de/testreports/SUSE:Maintenance:22399:262470/install_logs/sles_15-SP3_aarch64.log
sles_15-SP3_s390x => passed: http://qam.suse.de/testreports/SUSE:Maintenance:22399:262470/install_logs/sles_15-SP3_s390x.log
sles_15-SP3_x86_64 => passed: http://qam.suse.de/testreports/SUSE:Maintenance:22399:262470/install_logs/sles_15-SP3_x86_64.log
but if we open the links, we can see that the operation was a "nothing to do".
Digging into openQA, it seems that the installation was skipped but the test was as marked as passed:
https://openqa.suse.de/tests/7994974#step/update_install/42
https://openqa.suse.de/tests/7994977#step/update_install/31
I have therefore a couple of questions:
- are we doing installation tests only for some modules? (e.g. this one is for sle-module-development-tools)
- why a skipped installation test is marked as passed?
Thanks