Project

General

Profile

Actions

action #105124

open

[qe-core] qam-incidentinstall is green even when installation is skipped

Added by pstivanin about 2 years ago. Updated about 1 year ago.

Status:
Workable
Priority:
Normal
Assignee:
-
Category:
Bugs in existing tests
Target version:
-
Start date:
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Hello,
tt seems that openQA treats as valid result (i.e. green) an empty installation test.
For example, looking at qam-incidentinstall of this MU https://openqa.suse.de/tests/overview?distri=sle&version=15-SP3&build=%3A22399%3Arpmlint&groupid=367, we can see that it's green for all 4 architectures. Even the generated report looks good:

sled_15-SP3_x86_64 => passed: http://qam.suse.de/testreports/SUSE:Maintenance:22399:262470/install_logs/sled_15-SP3_x86_64.log
sles_15-SP3_aarch64 => passed: http://qam.suse.de/testreports/SUSE:Maintenance:22399:262470/install_logs/sles_15-SP3_aarch64.log
sles_15-SP3_s390x => passed: http://qam.suse.de/testreports/SUSE:Maintenance:22399:262470/install_logs/sles_15-SP3_s390x.log
sles_15-SP3_x86_64 => passed: http://qam.suse.de/testreports/SUSE:Maintenance:22399:262470/install_logs/sles_15-SP3_x86_64.log

but if we open the links, we can see that the operation was a "nothing to do".
Digging into openQA, it seems that the installation was skipped but the test was as marked as passed:

https://openqa.suse.de/tests/7994974#step/update_install/42
https://openqa.suse.de/tests/7994977#step/update_install/31

I have therefore a couple of questions:

  1. are we doing installation tests only for some modules? (e.g. this one is for sle-module-development-tools)
  2. why a skipped installation test is marked as passed?

Thanks


Related issues 1 (0 open1 closed)

Related to openQA Tests - action #101938: [qe-core] update_install failed to install update but test passedResolvedapappas

Actions
Actions #1

Updated by coolo about 2 years ago

  • Project changed from openQA Project to openQA Tests

This is something that should be discussed with the test maintainer of update_install:
# Maintainer: Ondřej Súkup osukup@suse.cz, Anton Pappas apappas@suse.com

Actions #2

Updated by apappas about 2 years ago

  • Related to action #101938: [qe-core] update_install failed to install update but test passed added
Actions #3

Updated by maritawerner about 2 years ago

  • Subject changed from qam-incidentinstall is green even when installation is skipped to [qe-core] qam-incidentinstall is green even when installation is skipped
Actions #4

Updated by okurz about 2 years ago

  • Category set to Bugs in existing tests
Actions #5

Updated by tjyrinki_suse about 2 years ago

  • Status changed from New to Workable
  • Start date deleted (2022-01-20)
Actions #6

Updated by slo-gin about 1 year ago

This ticket was set to Normal priority but was not updated within the SLO period. Please consider picking up this ticket or just set the ticket to the next lower priority.

Actions

Also available in: Atom PDF