[sle][functional][12sp4][u]test fails in docker - needs to add container module / how do we want to execute extra tests that rely on a registered image for SLE12?
|Target version:||QA - future|
openQA test in scenario sle-12-SP4-Server-DVD-x86_64-extra_tests_in_textmode@64bit fails in
Fails since (at least) Build 0095
- Schedule the docker module in the "proxy_SCC+allmodules" scenario for SLE12 product validation and not in extra_tests_in_textmode
- Ensure the module is still scheduled for maintenance tests
- Ensure the module is still scheduled as "extra_tests" for SLE15
- Remove the hack from https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/e70ba1caa4403c93f230ed94279c0971d7f8f933/lib/registration.pm#L671 to not add a repo, but installing docker from the proper repository.
Always latest result in this scenario: latest
Reasoning for the current suggestions¶
The modules within SLE12 are developed and maintained separately from SLES using the maintenance process. Therefore we also use the module content of publically published repos and not any internal copy which is not developed in sync with any build of SLES anyway. Therefore we need to select a different approach than for SLE15 anyway. We already do that approach within "proxy_SCC+allmodules", e.g. for "puppet" or "salt" from ASMM. Publishing a "registered" image could mislead us to think that the module content would be consistent with the build of SLES12 but there is no hard relation between the both.
#1 Updated by okurz almost 2 years ago
- Description updated (diff)
- Category changed from Bugs in existing tests to New test
Actually it is a new test that could have never worked because by default we are using an unregistered SLE image for SLE12 tests. For now I propose to skip the scheduling of the test modules for SLE12SP4: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4614
#4 Updated by okurz almost 2 years ago
- Subject changed from [sle][functional][12sp4]test fails in docker - needs to add container module / how do we want to execute extra tests that rely on a registered image for SLE12? to [sle][functional][12sp4][u]test fails in docker - needs to add container module / how do we want to execute extra tests that rely on a registered image for SLE12?
#6 Updated by okurz almost 2 years ago
- O1: Just register the whole system and add the module for just one test module -> deregister again after the test module
- O2: Publish the hdd image from a test suite that registers the system and add a new test scenario like "extra_tests_registered" where we would schedule the docker module -> Con Quite different test flow for SLE12 vs. SLE15 as well as openSUSE
- O3: Manually add the one repo folder from openqa.suse.de -> Con Might not seem legit to everybody because we are not directly using the module as registered over SCC, Pro Easy way to not need another test suite and not mangle with registration
#9 Updated by okurz almost 2 years ago
PR merged but we should keep this ticket open and resolve the open points from the discussion on github:
Please don't do it in this way. QAM tests have already container module added during installation.
See https://openqa.suse.de/tests/1604714 MRU_ADDONS=sdk,tcm,wsm,we,contm
So basically there is no need to add special repo for SLE12-SP3(not sure about SP4).
@czerw, please be aware that the repo will only be added if package docker is not found. When container module is already added, the repo will not be
added. So it may work as expected.
But, I am still skeptical of this change. Now the reason why we need to add that repo is shadowed (to trick that we are using an image with
unregistered SUT. An scenario where the customer also could not install docker). This tricky added repo may affect the following test modules on this
test suite and people may forget about this and will probably understand/change this wrong. I would still prefer to use an image with registered SUT,
so we avoid false negatives and confusion.
#10 Updated by okurz almost 2 years ago
- Due date deleted (
- Assignee deleted (
- Target version changed from Milestone 15 to future
https://openqa.suse.de/tests/1609882#step/docker/70 works. Now … discuss! :)
#11 Updated by pcervinka almost 2 years ago
Oliver, thank you for update. I re-checked SLE-SP3 run https://openqa.suse.de/tests/1611021#step/docker/2 and it works as expected and maintenance test is not impacted. So thumbs up :)
#12 Updated by okurz almost 2 years ago
good to hear that, thank you. Next to the open point about "shouldn't we use repos over proxy scc for SLE12" I have one more open point which I also mentioned to pluskalm today: "Can we find a better long-term solution for tests that depend on the LTSS/non-LTSS state of the product" so that we do not have to update versions for tests like these just because a product goes into LTSS?
#18 Updated by okurz about 1 year ago
- Parent task deleted (
I don't understand the two new tasks you added. Why do you think we should create "registered" images?
Removing parent job and adding back as related as for now I think we can close the epic which was about "cover every module" and I think we do that well enough for installation as well as migration.
#21 Updated by okurz about 1 year ago
- Description updated (diff)
- Assignee deleted (
ok, fine. So basically what I proposed in #33184#note-6 . But, I discussed with SLindoMansilla in person and we found a new way, updated the description to use the "proxy_SCC+allmodules" scenario for SLE12 as the point where the test module should be executed.