Project

General

Profile

Actions

action #19566

closed

[sle][functional]New test scenario: minimal+proxy_SCC-postreg_suseconnect_scc

Added by okurz almost 7 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
High
Assignee:
-
Category:
New test
Start date:
2017-06-04
Due date:
2017-11-08
% Done:

0%

Estimated time:
Difficulty:

Description

Acceptance criteria

  • AC1: Have a test scenario for SLE15 to test registration using SUSEConnect against proxy-SCC

Tasks

  • Create the test module to register using SUSEConnect ======== done
  • Create for a test scenario for SLE15 ======== done

Related issues 1 (0 open1 closed)

Related to openQA Tests - action #26984: [sle][functional][y] New test scenario: minimal+proxy_SCC-postreg_yast_sccRejectedriafarov2017-10-24

Actions
Actions #1

Updated by Anonymous almost 7 years ago

  • Status changed from New to In Progress
  • Assignee set to Anonymous
Actions #2

Updated by Anonymous almost 7 years ago

  • Status changed from In Progress to Resolved

The scenario is gone. Neither in openqa.suse.de nor openqa.opensuse.org testsuite database any more.

Actions #3

Updated by okurz almost 7 years ago

  • Status changed from Resolved to In Progress

yi, please don't do that. The testsuite "minimal+base+sdk+proxy_SCC-postreg" still exists in https://openqa.suse.de/admin/test_suites and if it would be gone we must not just accept this but be very careful that we are not loosing anything from our test coverage. I already added a description to the test suite. I don't know why it's not in test development anymore but feel free to add it back. Could you check the test for the tasks I suggested in the original ticket description?

Actions #4

Updated by Anonymous over 6 years ago

The scenario is called minimal+base+sdk+proxy_SCC-postreg, belonging to job group Functional: Server.
Here is an example testrun: https://openqa.suse.de/tests/1058256

Actions #5

Updated by Anonymous over 6 years ago

For SLE15 it already exists: https://openqa.suse.de/tests/1119787

Actions #6

Updated by Anonymous over 6 years ago

Local testrun: f146.suse.de/tests/1016

Actions #7

Updated by okurz over 6 years ago

please update tickets "In Progress" at latest EOB (end of business day) with either having it resolved - of course preferred ;-) - or with a clear comment why it was not finishable within the day.

Actions #9

Updated by okurz over 6 years ago

  • Related to action #25352: [sle][functional]test fails in yast_scc - scenario minimal+base+sdk+proxy_SCC-postreg added
Actions #10

Updated by okurz over 6 years ago

  • Subject changed from New test scenario: minimal+base+sdk+fakescc-postreg to New test scenario: minimal+base+sdk+proxy_SCC-postreg (was: minimal+base+sdk+fakescc-postreg)
  • Description updated (diff)
  • Priority changed from Normal to High

With SLE15 and more product changes this scenario gets more important so IMHO we should bump prio.

Updated the description with acceptance criteria and updated tasks. That should help to go forward.

Actions #12

Updated by Anonymous over 6 years ago

PR updated.

Actions #13

Updated by okurz over 6 years ago

  • Subject changed from New test scenario: minimal+base+sdk+proxy_SCC-postreg (was: minimal+base+sdk+fakescc-postreg) to [sle][functional]New test scenario: minimal+base+sdk+proxy_SCC-postreg (was: minimal+base+sdk+fakescc-postreg)
Actions #14

Updated by Anonymous over 6 years ago

  • Due date set to 2017-10-11
Actions #15

Updated by okurz over 6 years ago

  • Target version set to Milestone 11
Actions #16

Updated by Anonymous over 6 years ago

  • Assignee deleted (Anonymous)
Actions #18

Updated by Anonymous over 6 years ago

  • Assignee set to Anonymous
Actions #19

Updated by Anonymous over 6 years ago

A new test scenario will be created. Verification testrun: http://f146.suse.de/tests/1352 http://f146.suse.de/tests/1353

Actions #20

Updated by okurz over 6 years ago

As discussed with yi and riafarov I reminded about the suggestions in the ticket description. I recommend to try a test plan that should look as simple as:

  • boot_to_desktop
  • suseconnect_scc

and another test suite with:

  • boot_to_desktop
  • yast_scc

If necessary add additional modules - but only if necessary - e.g. sle15_workarounds or consoletest_setup or hostname. If the above is done - but only after it's done - you can add more test modules, e.g. "zypper_lr" to list the enabled repositories.

Actions #22

Updated by okurz over 6 years ago

  • Due date changed from 2017-10-11 to 2017-10-25

have to use double ticks in script_run 'SUSEConnect -r $reg_code'; and use assert_script_run -> assert_script_run "SUSEConnect -r $reg_code";

@riafarov, @yi

Actions #23

Updated by Anonymous over 6 years ago

PR updated. Verification testruns are: http://f146.suse.de/tests/1454 http://f146.suse.de/tests/1455
At the moment the test fails because the repos couldn't be refreshed.

Actions #24

Updated by Anonymous over 6 years ago

Testsuites are added. However cannot be executed yet because qcow images are not available.

Actions #25

Updated by okurz over 6 years ago

failed as incomplete on zkvm: https://openqa.suse.de/tests/1209421

Please make sure to not keep scenarios that end up incomplete

Actions #26

Updated by okurz over 6 years ago

https://openqa.suse.de/tests/1209422#step/boot_to_desktop/2 is the failing scenario you added for zVM. This cannot work as zVM instances can not boot disk images. Also the scenario does not state 'START_AFTER_TEST. This won't work. I suggest to use the "test development" job group for experiments.

Actions #27

Updated by Anonymous over 6 years ago

  • Status changed from In Progress to Feedback

Because qcow images are missing which results incomplete testruns, I changed the ticket status to feedback.

Actions #28

Updated by okurz over 6 years ago

Keep in mind though that jobs should not incomplete. If that is the case then they are incorrectly configured. Rather they should cancel if the parent job fails to provide a qcow image, which they do: https://openqa.suse.de/tests/1213106
So ok up to now. Also most recent build 303.1 could not provide a qcow image so understandable that we are blocked by this still.
A workaround is to retrigger an older build which still worked.

Actions #29

Updated by Anonymous over 6 years ago

  • Status changed from Feedback to In Progress
Actions #30

Updated by okurz over 6 years ago

@yi please update the ticket description with more recent tasks and an effort estimation according to the template as we also have it in other tickets.

Actions #31

Updated by Anonymous over 6 years ago

Atm the test fails at boot_to_desktop since it didn't boot to desktop but only text mode:
https://openqa.suse.de/tests/1226927#step/boot_to_desktop/6

However I couldn't reproduce it locally, reasons: when I copied the job, it is scheduled after skip_registration, which already fails at first_boot, so that the test couldn't continue.

Besides that, can anyone tell me the reason for scheduling it after skip_registration?

Actions #32

Updated by Anonymous over 6 years ago

  • Subject changed from [sle][functional]New test scenario: minimal+base+sdk+proxy_SCC-postreg (was: minimal+base+sdk+fakescc-postreg) to [sle][functional]New test scenario: minimal+proxy_SCC-postreg (was: minimal+base+sdk+fakescc-postreg)
  • Description updated (diff)
Actions #33

Updated by Anonymous over 6 years ago

  • Description updated (diff)
Actions #34

Updated by Anonymous over 6 years ago

Trying with DESKTOP=textmode:
geekotest@shaka:~> /usr/share/openqa/script/clone_job.pl --from https://openqa.suse.de --host https://openqa.suse.de --skip-download --skip-deps 1226927 DESKTOP=textmode _GROUP=0 TEST=postreg_scc_yxu_tryout_with_desktop_textmode
Created job #1227554: sle-15-Installer-DVD-x86_64-Build310.1-minimal+proxy_SCC-postreg_SUSEconnect@64bit

Actions #35

Updated by Anonymous over 6 years ago

There's dependency issues between modules, the source code should be fixed.

Actions #36

Updated by okurz over 6 years ago

Yes, I think either use @mnowak's code or in case of suseconnect_scc.pm instead of

foreach (values %registration::SLE15_MODULES) {
    assert_script_run "SUSEConnect -p sle-module-" . lc($_) . "/$version/$arch";
}

it should be

foreach (split(',', $registration::SLE15_DEFAULT_MODULES{get_required_var('SLE_PRODUCT')})) {
    assert_script_run "SUSEConnect -p sle-module-" . lc($registration::SLE15_MODULES{$_}) . "/$version/$arch";
}

(code not verified)

to

  1. use the default modules only
  2. support all products
  3. use the correct order of modules
Actions #37

Updated by okurz over 6 years ago

"time inestimably": Easy suggestion: When you can't find a time estimation stop work immediately. What's the sense to start a work when you don't know if you can ever finish? I think you can find an effort estimation (not duration) with a <optimistic>-<pessimistic> range.

"…because of other dependent issues": which ones?

yi: issue was mentioned in comment 31, now resolved.

Actions #38

Updated by Anonymous over 6 years ago

  • Description updated (diff)
Actions #39

Updated by Anonymous over 6 years ago

PTF submitted. Now I will try to adapt the code form sccreg.pm.

Actions #40

Updated by Anonymous over 6 years ago

I had a look of lib/suseconnect_register.pm, it will be more complicated to call one function command_register from there, since it uses SCC_ADDONS as the queue of modules to be registered and the modules are also kept in hash.

Another look at jeos/sccreg.pm: it is very similar to console/suseconnect_scc.pm, the only difference is that the modules for jeos are: ('basesystem', 'scripting', 'legacy'), so there should be a way to reduce code redundancy here actually. However this PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3776 should fix the problem for now.

Verification testruns on osd: http://openqa.suse.de/tests/1230963 http://openqa.suse.de/tests/1230964

Actions #41

Updated by SLindoMansilla over 6 years ago

  • Description updated (diff)

yxi, how can I help you with this?
I see that the current step to do is fixing the problem about missing yast2 on SLE15 minimal for the module yast_scc.

Actions #42

Updated by okurz over 6 years ago

the scenario "yast_scc" can not work this way as the HDD image does not provide yast (and should not). I removed the scenario from the functional job group and instead moved it to Test Development: SLE 15 (64bit only). The suseconnect scenario for zkvm was incomplete because there is no "skip_registration@zkvm", only "skip_registration@s390x-kvm". Added the corresponding scenario to Test Development: SLE 15 as well.

Actions #43

Updated by SLindoMansilla over 6 years ago

  • Related to action #26042: [sle][functional][sle15] new system role - "textmode" added
Actions #44

Updated by SLindoMansilla over 6 years ago

  • Related to action #23408: [sle][functional]Use single test suite for create_hdd_textmode on all architectures added
Actions #45

Updated by SLindoMansilla over 6 years ago

Providing a basis: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3781
A lot to be improved yet.

Actions #46

Updated by Anonymous over 6 years ago

Oliver, the content of SLE15_MODULES and SLE15_DEFAULT_MODULES are different, and the naming matters: http://openqa.suse.de/tests/1230963#step/suseconnect_scc/16 . Could you explain why you are using SLE15_DEFAULT_MODULES instead of SLE15_MODULES? And I think the easiest way is still to define the modules just in an array.

Actions #47

Updated by Anonymous over 6 years ago

  • Subject changed from [sle][functional]New test scenario: minimal+proxy_SCC-postreg (was: minimal+base+sdk+fakescc-postreg) to [sle][functional]New test scenario: minimal+proxy_SCC-postreg_suseconnect_scc
  • Description updated (diff)
Actions #48

Updated by Anonymous over 6 years ago

Hi Sergio, I have separated the scenario minimal+proxy_SCC-postreg_yast_scc in another ticket: https://progress.opensuse.org/issues/26984
And since you are already working on it, I assigned it to you. Thanks.

Actions #49

Updated by Anonymous over 6 years ago

  • Related to deleted (action #25352: [sle][functional]test fails in yast_scc - scenario minimal+base+sdk+proxy_SCC-postreg )
Actions #50

Updated by Anonymous over 6 years ago

  • Related to deleted (action #26042: [sle][functional][sle15] new system role - "textmode")
Actions #51

Updated by Anonymous over 6 years ago

  • Related to deleted (action #23408: [sle][functional]Use single test suite for create_hdd_textmode on all architectures)
Actions #52

Updated by okurz over 6 years ago

  • Related to action #26984: [sle][functional][y] New test scenario: minimal+proxy_SCC-postreg_yast_scc added
Actions #53

Updated by okurz over 6 years ago

  • Due date changed from 2017-10-25 to 2017-11-08

Yes, the ticket split makes sense now. So with this I feel confident that we can resolve the ticket within sprint 3 :) As agreed with PO -> sprint 3

Actions #54

Updated by Anonymous over 6 years ago

Last verification on osd failed although on my local machine succeeded. New verification: http://openqa.suse.de/tests/1233670

Actions #55

Updated by Anonymous over 6 years ago

  • Status changed from In Progress to Resolved

Verification testrun succeeded.

Actions #56

Updated by okurz over 6 years ago

That's good! The last job within the QA SLE functional job group https://openqa.suse.de/tests/1233455 failed however to boot an image. Ideas why?

Actions #57

Updated by Anonymous over 6 years ago

Because the needle was lost? It couldn't boot. Should there some more parameters be specified? But on aarch64 it succeeded: https://openqa.suse.de/tests/1233047#step/boot_to_desktop/1

Actions #58

Updated by okurz over 6 years ago

https://openqa.suse.de/tests/1233455#step/boot_to_desktop/2 does not look like any needle was lost

Actions

Also available in: Atom PDF