Project

General

Profile

Actions
Copy link

action #43853

closed

coordination #40484: [qe-core][functional][epic] Move different checks to separate test suites not to affect other functional tests

[functional][y] Do not assert preselected modules in each scenario

Added by riafarov over 5 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
JERiveraMoya
Category:
New test
Target version:
SUSE QA - Milestone 23
Start date:
2018-11-15
Due date:
% Done:

0%

Estimated time:
Difficulty:

Description

Motivation

See motivation in the parent ticket.

Acceptance criteria

  1. Pre-selected modules are asserted only once per product for a single build
  2. Current coverage is not decreased (pre-selected modules are checked for all the products as before)

Suggestions

We can move this check to extended installer for SLES, but we also need to assure that for all other products we assert pre-selected modules including ones, which QSF virtual team is not responsible for (e.g. SLES 4SAP, SLED).

Simple solution would be to introduce activation variable and set it in all relevant scenarios.

Actions
Copy link
 #1

Updated by okurz over 5 years ago

  • Category set to New test
  • Target version set to Milestone 22
Actions
Copy link
 #2

Updated by JERiveraMoya over 5 years ago

  • Due date set to 2019-01-15
  • Status changed from Workable to Feedback
  • Assignee set to JERiveraMoya
Actions
Copy link
 #3

Updated by okurz over 5 years ago

  • Target version changed from Milestone 22 to Milestone 23
Actions
Copy link
 #4

Updated by JERiveraMoya over 5 years ago

It is curious that the image for Desktop group does not select the product at the beginning an add the modules later: https://openqa.suse.de/tests/2360635#step/scc_registration/12
is it intended ...? forget about it, it is the workstation module :) they select it in other scenario, for example: https://openqa.suse.de/tests/2360440#step/scc_registration/12 I am going to add it to the VR in the PR.

Actions
Copy link
 #5

Updated by JERiveraMoya over 5 years ago

  • Status changed from Feedback to In Progress

New test suites created. Waiting for next build.

Actions
Copy link
 #6

Updated by JERiveraMoya over 5 years ago

  • Status changed from In Progress to Feedback
Actions
Copy link
 #7

Updated by riafarov over 5 years ago

  • Due date changed from 2019-01-15 to 2019-01-29

offline installations is also one of the scenarios to have such checks enabled.

Actions
Copy link
 #8

Updated by okurz over 5 years ago

I understand the original ticket description in a way that we would need to be more strict about which modules need to be explicity selected because we do not want to conduct the dynamic check in every job run, right?

What I find confusing is that e06294fc8 by JERiveraMoya introduced the method verify_preselected_modules which I understand is actually doing more than just "verifying" but ensuring that the right modules are selected even if the module page is longer than a single page.

After https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6500 merged I am a little bit concerned we would break some other tests. Could you please crosscheck e.g. QAM tests or SLES4SAP, HPC, etc.?

Actions
Copy link
 #9

Updated by JERiveraMoya over 5 years ago

  • Yes, exactly, is for specifying all those checks in one test suite and not doing it in all of them.
  • We can address better verify_preselected_modules in future. It is only used this scroll in a different test (I remember we were testing some yast module) that is not in the installer. When for some of the product it will be insufficient doing the checking in one shot, then we can adapt this function in case it would not work.
  • Found kind of a break: PR: Increase timeout when not checking preselected modules
  • Explained what is the purpose of the test suite to @ldevulder.
  • New test suites scheduled fine: hpc ha sles4sap
Actions
Copy link
 #11

Updated by JERiveraMoya over 5 years ago

  • Status changed from In Progress to Resolved
Actions
Copy link
 #12

Updated by okurz over 5 years ago

  • Status changed from Resolved to In Progress
  • Assignee changed from JERiveraMoya to okurz
  • Priority changed from Normal to Urgent

Some more scenarios are failing because of missing needles which is critical as of now because people are waiting for the child jobs for SLE15SP1Beta2 validation. I could not reach you over IRC (lunch time, I guess) so I took the ticket for now and can "give it back" later.

Actions
Copy link
 #13

Updated by okurz over 5 years ago

https://openqa.suse.de/tests/2386707#step/scc_registration/10 failed, created new needle scc-beta-filter-unchecked-textmode-15sp1-20190118 . Scheduled new test https://openqa.suse.de/tests/2387142

Actions
Copy link
 #14

Updated by JERiveraMoya over 5 years ago

Thanks, yes, there was some differences between bold and not bold font among s390s and other in textmode. I think we have covered it.

Actions
Copy link
 #15

Updated by okurz over 5 years ago

Also in the yast configuration module after installation: https://openqa.suse.de/tests/2386228#step/addon_products_via_SCC_yast2/27

Created new needle addon_products_via_SCC_yast2-scc-beta-filter-unchecked-15sp1-20190118 and retriggered as https://openqa.suse.de/tests/2387285

Actions
Copy link
 #16

Updated by okurz over 5 years ago

  • Status changed from In Progress to Resolved
  • Assignee changed from okurz to JERiveraMoya
  • Priority changed from Urgent to Normal

Seems to look fine now with the new needles. There are still currently running jobs in SLE15SP1 build 144.3 but it seems they have all passed that step or are still scheduled and the new needles should be picked up.

Actions
Copy link
 #17

Updated by okurz over 2 years ago

  • Due date deleted (2019-01-29)
Actions
Copy link

Also available in: Atom PDF