Project

General

Profile

action #105124

[qe-core] qam-incidentinstall is green even when installation is skipped

Added by pstivanin 4 months ago. Updated 3 months ago.

Status:
Workable
Priority:
Normal
Assignee:
-
Category:
Bugs in existing tests
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Hello,
tt seems that openQA treats as valid result (i.e. green) an empty installation test.
For example, looking at qam-incidentinstall of this MU https://openqa.suse.de/tests/overview?distri=sle&version=15-SP3&build=%3A22399%3Arpmlint&groupid=367, we can see that it's green for all 4 architectures. Even the generated report looks good:

sled_15-SP3_x86_64 => passed: http://qam.suse.de/testreports/SUSE:Maintenance:22399:262470/install_logs/sled_15-SP3_x86_64.log
sles_15-SP3_aarch64 => passed: http://qam.suse.de/testreports/SUSE:Maintenance:22399:262470/install_logs/sles_15-SP3_aarch64.log
sles_15-SP3_s390x => passed: http://qam.suse.de/testreports/SUSE:Maintenance:22399:262470/install_logs/sles_15-SP3_s390x.log
sles_15-SP3_x86_64 => passed: http://qam.suse.de/testreports/SUSE:Maintenance:22399:262470/install_logs/sles_15-SP3_x86_64.log

but if we open the links, we can see that the operation was a "nothing to do".
Digging into openQA, it seems that the installation was skipped but the test was as marked as passed:

https://openqa.suse.de/tests/7994974#step/update_install/42
https://openqa.suse.de/tests/7994977#step/update_install/31

I have therefore a couple of questions:

  1. are we doing installation tests only for some modules? (e.g. this one is for sle-module-development-tools)
  2. why a skipped installation test is marked as passed?

Thanks


Related issues

Related to openQA Tests - action #101938: [qe-core] update_install failed to install update but test passedResolved

History

#1 Updated by coolo 4 months ago

  • Project changed from openQA Project to openQA Tests

This is something that should be discussed with the test maintainer of update_install:
# Maintainer: Ondřej Súkup osukup@suse.cz, Anton Pappas apappas@suse.com

#2 Updated by apappas 4 months ago

  • Related to action #101938: [qe-core] update_install failed to install update but test passed added

#3 Updated by maritawerner 4 months ago

  • Subject changed from qam-incidentinstall is green even when installation is skipped to [qe-core] qam-incidentinstall is green even when installation is skipped

#4 Updated by okurz 4 months ago

  • Category set to Bugs in existing tests

#5 Updated by tjyrinki_suse 3 months ago

  • Status changed from New to Workable
  • Start date deleted (2022-01-20)

Also available in: Atom PDF