action #43853
closedcoordination #40484: [qe-core][functional][epic] Move different checks to separate test suites not to affect other functional tests
[functional][y] Do not assert preselected modules in each scenario
0%
Description
Motivation¶
See motivation in the parent ticket.
Acceptance criteria¶
- Pre-selected modules are asserted only once per product for a single build
- Current coverage is not decreased (pre-selected modules are checked for all the products as before)
Suggestions¶
We can move this check to extended installer for SLES, but we also need to assure that for all other products we assert pre-selected modules including ones, which QSF virtual team is not responsible for (e.g. SLES 4SAP, SLED).
Simple solution would be to introduce activation variable and set it in all relevant scenarios.
Updated by okurz over 5 years ago
- Category set to New test
- Target version set to Milestone 22
Updated by JERiveraMoya over 5 years ago
- Due date set to 2019-01-15
- Status changed from Workable to Feedback
- Assignee set to JERiveraMoya
Updated by okurz over 5 years ago
- Target version changed from Milestone 22 to Milestone 23
Updated by JERiveraMoya over 5 years ago
It is curious that the image for Desktop group does not select the product at the beginning an add the modules later: https://openqa.suse.de/tests/2360635#step/scc_registration/12
is it intended ...? forget about it, it is the workstation module :) they select it in other scenario, for example: https://openqa.suse.de/tests/2360440#step/scc_registration/12 I am going to add it to the VR in the PR.
Updated by JERiveraMoya over 5 years ago
- Status changed from Feedback to In Progress
New test suites created. Waiting for next build.
Updated by JERiveraMoya over 5 years ago
- Status changed from In Progress to Feedback
Updated by riafarov over 5 years ago
- Due date changed from 2019-01-15 to 2019-01-29
offline installations is also one of the scenarios to have such checks enabled.
Updated by okurz over 5 years ago
I understand the original ticket description in a way that we would need to be more strict about which modules need to be explicity selected because we do not want to conduct the dynamic check in every job run, right?
What I find confusing is that e06294fc8 by JERiveraMoya introduced the method verify_preselected_modules
which I understand is actually doing more than just "verifying" but ensuring that the right modules are selected even if the module page is longer than a single page.
After https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6500 merged I am a little bit concerned we would break some other tests. Could you please crosscheck e.g. QAM tests or SLES4SAP, HPC, etc.?
Updated by JERiveraMoya over 5 years ago
- Yes, exactly, is for specifying all those checks in one test suite and not doing it in all of them.
- We can address better
verify_preselected_modules
in future. It is only used this scroll in a different test (I remember we were testing some yast module) that is not in the installer. When for some of the product it will be insufficient doing the checking in one shot, then we can adapt this function in case it would not work. - Found kind of a break: PR: Increase timeout when not checking preselected modules
- Explained what is the purpose of the test suite to @ldevulder.
- New test suites scheduled fine: hpc ha sles4sap
Updated by JERiveraMoya over 5 years ago
- Status changed from Feedback to In Progress
Updated by JERiveraMoya over 5 years ago
- Status changed from In Progress to Resolved
Updated by okurz over 5 years ago
- Status changed from Resolved to In Progress
- Assignee changed from JERiveraMoya to okurz
- Priority changed from Normal to Urgent
Some more scenarios are failing because of missing needles which is critical as of now because people are waiting for the child jobs for SLE15SP1Beta2 validation. I could not reach you over IRC (lunch time, I guess) so I took the ticket for now and can "give it back" later.
Updated by okurz over 5 years ago
https://openqa.suse.de/tests/2386707#step/scc_registration/10 failed, created new needle scc-beta-filter-unchecked-textmode-15sp1-20190118 . Scheduled new test https://openqa.suse.de/tests/2387142
Updated by JERiveraMoya over 5 years ago
Thanks, yes, there was some differences between bold and not bold font among s390s and other in textmode. I think we have covered it.
Updated by okurz over 5 years ago
Also in the yast configuration module after installation: https://openqa.suse.de/tests/2386228#step/addon_products_via_SCC_yast2/27
Created new needle addon_products_via_SCC_yast2-scc-beta-filter-unchecked-15sp1-20190118 and retriggered as https://openqa.suse.de/tests/2387285
Updated by okurz over 5 years ago
- Status changed from In Progress to Resolved
- Assignee changed from okurz to JERiveraMoya
- Priority changed from Urgent to Normal
Seems to look fine now with the new needles. There are still currently running jobs in SLE15SP1 build 144.3 but it seems they have all passed that step or are still scheduled and the new needles should be picked up.