action #33184

[sle][functional][12sp4][u]test fails in docker - needs to add container module / how do we want to execute extra tests that rely on a registered image for SLE12?

Added by okurz almost 2 years ago. Updated 6 months ago.

Status:RejectedStart date:13/03/2018
Priority:LowDue date:
Assignee:-% Done:

0%

Category:New test
Target version:QA - future
Difficulty:
Duration:

Description

Observation

openQA test in scenario sle-12-SP4-Server-DVD-x86_64-extra_tests_in_textmode@64bit fails in
docker

Reproducible

Fails since (at least) Build 0095

Suggestions

Further details

Always latest result in this scenario: latest

Reasoning for the current suggestions

The modules within SLE12 are developed and maintained separately from SLES using the maintenance process. Therefore we also use the module content of publically published repos and not any internal copy which is not developed in sync with any build of SLES anyway. Therefore we need to select a different approach than for SLE15 anyway. We already do that approach within "proxy_SCC+allmodules", e.g. for "puppet" or "salt" from ASMM. Publishing a "registered" image could mislead us to think that the module content would be consistent with the build of SLES12 but there is no hard relation between the both.


Related issues

Related to openQA Tests - action #9466: [sle][functional][epic][medium][u]Modules - Containers: T... Resolved 12/06/2015

History

#1 Updated by okurz almost 2 years ago

  • Description updated (diff)
  • Category changed from Bugs in existing tests to New test

Actually it is a new test that could have never worked because by default we are using an unregistered SLE image for SLE12 tests. For now I propose to skip the scheduling of the test modules for SLE12SP4: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4614

#2 Updated by okurz almost 2 years ago

  • Subject changed from [sle][functional][12sp4]test fails in docker - needs to add container module to [sle][functional][12sp4]test fails in docker - needs to add container module / how do we want to execute extra tests that rely on a registered image for SLE12?

#3 Updated by okurz almost 2 years ago

Need to discuss in a daily with the team to decide.

#4 Updated by okurz almost 2 years ago

  • Subject changed from [sle][functional][12sp4]test fails in docker - needs to add container module / how do we want to execute extra tests that rely on a registered image for SLE12? to [sle][functional][12sp4][u]test fails in docker - needs to add container module / how do we want to execute extra tests that rely on a registered image for SLE12?

#5 Updated by okurz almost 2 years ago

  • Parent task set to #9466

#6 Updated by okurz almost 2 years ago

Different options:

  • O1: Just register the whole system and add the module for just one test module -> deregister again after the test module
  • O2: Publish the hdd image from a test suite that registers the system and add a new test scenario like "extra_tests_registered" where we would schedule the docker module -> Con Quite different test flow for SLE12 vs. SLE15 as well as openSUSE
  • O3: Manually add the one repo folder from openqa.suse.de -> Con Might not seem legit to everybody because we are not directly using the module as registered over SCC, Pro Easy way to not need another test suite and not mangle with registration

#8 Updated by okurz almost 2 years ago

  • Due date changed from 10/04/2018 to 22/04/2018
  • Status changed from In Progress to Feedback

waiting for PR merged, should be not too much effort in next sprint.

#9 Updated by okurz almost 2 years ago

PR merged but we should keep this ticket open and resolve the open points from the discussion on github:

czerw:


Please don't do it in this way. QAM tests have already container module added during installation.
See https://openqa.suse.de/tests/1604714 MRU_ADDONS=sdk,tcm,wsm,we,contm
So basically there is no need to add special repo for SLE12-SP3(not sure about SP4).

SergioAtSUSE:


@czerw, please be aware that the repo will only be added if package docker is not found. When container module is already added, the repo will not be
added. So it may work as expected.

But, I am still skeptical of this change. Now the reason why we need to add that repo is shadowed (to trick that we are using an image with
unregistered SUT. An scenario where the customer also could not install docker). This tricky added repo may affect the following test modules on this
test suite and people may forget about this and will probably understand/change this wrong. I would still prefer to use an image with registered SUT,
so we avoid false negatives and confusion.

#10 Updated by okurz almost 2 years ago

  • Due date deleted (22/04/2018)
  • Assignee deleted (okurz)
  • Target version changed from Milestone 15 to future

#11 Updated by pcervinka almost 2 years ago

Oliver, thank you for update. I re-checked SLE-SP3 run https://openqa.suse.de/tests/1611021#step/docker/2 and it works as expected and maintenance test is not impacted. So thumbs up :)

#12 Updated by okurz almost 2 years ago

good to hear that, thank you. Next to the open point about "shouldn't we use repos over proxy scc for SLE12" I have one more open point which I also mentioned to pluskalm today: "Can we find a better long-term solution for tests that depend on the LTSS/non-LTSS state of the product" so that we do not have to update versions for tests like these just because a product goes into LTSS?

#13 Updated by okurz over 1 year ago

  • Target version changed from future to future

#14 Updated by SLindoMansilla over 1 year ago

  • Status changed from Feedback to Workable

I suspect this in feedback ticket is going nowhere without an assignee.

#15 Updated by okurz over 1 year ago

  • Status changed from Workable to Feedback
  • Assignee set to SLindoMansilla

We should discuss the open points from #33184#note-9 first before we can continue.

#16 Updated by SLindoMansilla over 1 year ago

I agree, I will take a look after the two urgent tickets, when I finish the one about journalctl.

#17 Updated by SLindoMansilla about 1 year ago

  • Description updated (diff)
  • Status changed from Feedback to Workable

Should I create 2 new tickets for the tasks?
Is this ticket workable again?

#18 Updated by okurz about 1 year ago

  • Parent task deleted (#9466)

I don't understand the two new tasks you added. Why do you think we should create "registered" images?

Removing parent job and adding back as related as for now I think we can close the epic which was about "cover every module" and I think we do that well enough for installation as well as migration.

#19 Updated by okurz about 1 year ago

  • Related to action #9466: [sle][functional][epic][medium][u]Modules - Containers: Test the SLE module for containers added

#20 Updated by SLindoMansilla about 1 year ago

Because of this: https://progress.opensuse.org/issues/33184#note-9

We are testing an artificial case. And that was from my point of view the best approach.

#21 Updated by okurz about 1 year ago

  • Description updated (diff)
  • Assignee deleted (SLindoMansilla)

ok, fine. So basically what I proposed in #33184#note-6 . But, I discussed with SLindoMansilla in person and we found a new way, updated the description to use the "proxy_SCC+allmodules" scenario for SLE12 as the point where the test module should be executed.

#22 Updated by okurz 12 months ago

  • Priority changed from Normal to Low

let's focus more on improving our current tests and workflows first. Putting to "holding tank" :)

#23 Updated by SLindoMansilla 6 months ago

  • Status changed from Workable to Rejected

Already tested, and there is a bug: https://bugzilla.suse.com/show_bug.cgi?id=1123502

Also available in: Atom PDF