action #33184
closed[sle][functional][12sp4][u]test fails in docker - needs to add container module / how do we want to execute extra tests that rely on a registered image for SLE12?
0%
Description
Observation¶
openQA test in scenario sle-12-SP4-Server-DVD-x86_64-extra_tests_in_textmode@64bit fails in
docker
Reproducible¶
Fails since (at least) Build 0095
Suggestions¶
- Schedule the docker module in the "proxy_SCC+allmodules" scenario for SLE12 product validation and not in extra_tests_in_textmode
- Ensure the module is still scheduled for maintenance tests
- Ensure the module is still scheduled as "extra_tests" for SLE15
- Remove the hack from https://github.com/os-autoinst/os-autoinst-distri-opensuse/blob/e70ba1caa4403c93f230ed94279c0971d7f8f933/lib/registration.pm#L671 to not add a repo, but installing docker from the proper repository.
Further details¶
Always latest result in this scenario: latest
Reasoning for the current suggestions¶
The modules within SLE12 are developed and maintained separately from SLES using the maintenance process. Therefore we also use the module content of publically published repos and not any internal copy which is not developed in sync with any build of SLES anyway. Therefore we need to select a different approach than for SLE15 anyway. We already do that approach within "proxy_SCC+allmodules", e.g. for "puppet" or "salt" from ASMM. Publishing a "registered" image could mislead us to think that the module content would be consistent with the build of SLES12 but there is no hard relation between the both.
Updated by okurz over 6 years ago
- Description updated (diff)
- Category changed from Bugs in existing tests to New test
Actually it is a new test that could have never worked because by default we are using an unregistered SLE image for SLE12 tests. For now I propose to skip the scheduling of the test modules for SLE12SP4: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4614
Updated by okurz over 6 years ago
- Subject changed from [sle][functional][12sp4]test fails in docker - needs to add container module to [sle][functional][12sp4]test fails in docker - needs to add container module / how do we want to execute extra tests that rely on a registered image for SLE12?
Updated by okurz over 6 years ago
Need to discuss in a daily with the team to decide.
Updated by okurz over 6 years ago
- Subject changed from [sle][functional][12sp4]test fails in docker - needs to add container module / how do we want to execute extra tests that rely on a registered image for SLE12? to [sle][functional][12sp4][u]test fails in docker - needs to add container module / how do we want to execute extra tests that rely on a registered image for SLE12?
Updated by okurz over 6 years ago
Different options:
- O1: Just register the whole system and add the module for just one test module -> deregister again after the test module
- O2: Publish the hdd image from a test suite that registers the system and add a new test scenario like "extra_tests_registered" where we would schedule the docker module -> Con Quite different test flow for SLE12 vs. SLE15 as well as openSUSE
- O3: Manually add the one repo folder from openqa.suse.de -> Con Might not seem legit to everybody because we are not directly using the module as registered over SCC, Pro Easy way to not need another test suite and not mangle with registration
Updated by okurz over 6 years ago
Updated by okurz over 6 years ago
- Due date changed from 2018-04-10 to 2018-04-22
- Status changed from In Progress to Feedback
waiting for PR merged, should be not too much effort in next sprint.
Updated by okurz over 6 years ago
PR merged but we should keep this ticket open and resolve the open points from the discussion on github:
czerw:
Please don't do it in this way. QAM tests have already container module added during installation.
See https://openqa.suse.de/tests/1604714 MRU_ADDONS=sdk,tcm,wsm,we,contm
So basically there is no need to add special repo for SLE12-SP3(not sure about SP4).
SergioAtSUSE:
@czerw, please be aware that the repo will only be added if package docker is not found. When container module is already added, the repo will not be
added. So it may work as expected.
But, I am still skeptical of this change. Now the reason why we need to add that repo is shadowed (to trick that we are using an image with
unregistered SUT. An scenario where the customer also could not install docker). This tricky added repo may affect the following test modules on this
test suite and people may forget about this and will probably understand/change this wrong. I would still prefer to use an image with registered SUT,
so we avoid false negatives and confusion.
Updated by okurz over 6 years ago
- Due date deleted (
2018-04-22) - Assignee deleted (
okurz) - Target version changed from Milestone 15 to future
https://openqa.suse.de/tests/1609882#step/docker/70 works. Now … discuss! :)
Updated by pcervinka over 6 years ago
Oliver, thank you for update. I re-checked SLE-SP3 run https://openqa.suse.de/tests/1611021#step/docker/2 and it works as expected and maintenance test is not impacted. So thumbs up :)
Updated by okurz over 6 years ago
good to hear that, thank you. Next to the open point about "shouldn't we use repos over proxy scc for SLE12" I have one more open point which I also mentioned to pluskalm today: "Can we find a better long-term solution for tests that depend on the LTSS/non-LTSS state of the product" so that we do not have to update versions for tests like these just because a product goes into LTSS?
Updated by SLindoMansilla about 6 years ago
- Status changed from Feedback to Workable
I suspect this in feedback ticket is going nowhere without an assignee.
Updated by okurz about 6 years ago
- Status changed from Workable to Feedback
- Assignee set to SLindoMansilla
We should discuss the open points from #33184#note-9 first before we can continue.
Updated by SLindoMansilla about 6 years ago
I agree, I will take a look after the two urgent tickets, when I finish the one about journalctl.
Updated by SLindoMansilla almost 6 years ago
- Description updated (diff)
- Status changed from Feedback to Workable
Should I create 2 new tickets for the tasks?
Is this ticket workable again?
Updated by okurz almost 6 years ago
- Parent task deleted (
#9466)
I don't understand the two new tasks you added. Why do you think we should create "registered" images?
Removing parent job and adding back as related as for now I think we can close the epic which was about "cover every module" and I think we do that well enough for installation as well as migration.
Updated by okurz almost 6 years ago
- Related to coordination #9466: [sle][functional][epic][medium][u]Modules - Containers: Test the SLE module for containers added
Updated by SLindoMansilla almost 6 years ago
Because of this: https://progress.opensuse.org/issues/33184#note-9
We are testing an artificial case. And that was from my point of view the best approach.
Updated by okurz almost 6 years ago
- Description updated (diff)
- Assignee deleted (
SLindoMansilla)
ok, fine. So basically what I proposed in #33184#note-6 . But, I discussed with SLindoMansilla in person and we found a new way, updated the description to use the "proxy_SCC+allmodules" scenario for SLE12 as the point where the test module should be executed.
Updated by okurz over 5 years ago
- Priority changed from Normal to Low
let's focus more on improving our current tests and workflows first. Putting to "holding tank" :)
Updated by SLindoMansilla over 5 years ago
- Status changed from Workable to Rejected
Already tested, and there is a bug: https://bugzilla.suse.com/show_bug.cgi?id=1123502