QA » openQA Project » openQA Tests

after thinking how to do that, i have only one solution, is an sys-admin/QA solution out of the scope of openqa-tests it self.

We should have a mirror/server that make update of the qcow(if manually or automated should be discussed) of the disks(qcow) that need to be updated. (automatically could be packages-conflicts).

With openqa itself, in the test-scenario (migrating before upgrading) technically, imho you have to boot first on the qcow, (no problem) but then you have to reattach the dvd, (this is done with qemu-backend at openqa launch. not during a test)to pass some grub parameter even edit the grub.cfg file to reboot on the disk-images, and i don't know if the disk passed by qemu should be there or not. So this approach sound to me too complicated and even there are too much dependencies that could break the test at beginning.

The first one could be a stable, general solution, but has to have more maintenance, but could have more generality, and scalable, since everyone could access the full-updated images, for running other tests.

I'm open for hints in both direction or new directions.

we already have a "Maintenance: Released Updates" job group (actually 2 per SP). You can schedule jobs in there to update the assets - this job group is scheduled daily. This means for migration we always have freshly patched qcows ready

checklist{

1. add/integrate updated sles-12-sp1.qcow: WIP
2. add/integrate updated sles-12-sp0.qcow:
3. add/integrate ..

if you have a registrated system (sles-12-sp0) and upgrade -> sles-12-sp2. this systems want to contact the registration proxy by default?
in this case, we should add a job on maintenance-group, to create a not registred qcow, otherwise we cannot use on migration scenario for sp2

Question: is SP-2 ready for accept scc request from registrated systems?

HOW_TO_REPRODUCE: Take the asset from qa-maintenance job-group, use this in a group, json file :

/usr/share/openqa/script/client --params update-12sp1.json jobs post
{
"ARCH" : "x86_64",
"BETA" : "1",
"BETA_SDK" : "1",
"BETA_WE" : "1",
"BUILD" : "1294",
"BUILD_HA" : "0196",
"BUILD_HA_GEO" : "0055",
"BUILD_SDK" : "0542",
"BUILD_WE" : "0154",
"DESKTOP" : "gnome",
"DISTRI" : "sle",
"DVD" : 1,
"FAKE_SCC_EMAIL" : "openqa@suse.de",
"FAKE_SCC_REGCODE" : "SLE-12-SP2-Server-DVD-x86_64-6gXVQP2tcAcEq3iD",
"FAKE_SCC_REGCODE_HA" : "SLE-12-SP2-Server-DVD-x86_64-6gXVQP2tcAcEq3iD",
"FAKE_SCC_REGCODE_HA_GEO" : "SLE-12-SP2-Server-DVD-x86_64-6gXVQP2tcAcEq3iD",
"FAKE_SCC_REGCODE_WE" : "SLE-12-SP2-Server-DVD-x86_64-6gXVQP2tcAcEq3iD",
"FAKE_SCC_URL" : "http://scc.openqa.suse.de",
"FLAVOR" : "Server-DVD",
"GNOME" : 1,
"HASLICENSE" : 1,
"HDDSIZEGB" : "40",
"NUMDISKS" : 1,
"NOIMAGES" : 1,
"HDD_1" : "SLE-Server-DVD-Updates-12-SP1-x86_64-default_registered_with_updates.qcow2",
"HOST" : "localhost",
"INSTLANG" : "en_US",
"ISO" : "SLE-12-SP2-Server-DVD-x86_64-Build1294-Media1.iso",
"ISO_MAXSIZE" : "4700372992",
"MACHINE" : "64bit",
"NOAUTOLOGIN" : 1,
"PACKAGETOINSTALL" : "x3270",
"QA_HEAD_REPO" : "http://dist.nue.suse.com/ibs/QA:/Head/SLE-12-SP2",
"REPO_0" : "SLE-12-SP2-Server-DVD-x86_64-Build1294-Media1",
"REPO_10" : "SLE-12-SP2-HA-POOL-x86_64-Build0196-Media1",
"REPO_11" : "SLE-12-SP2-HA-POOL-x86_64-Build0196-Media1.license",
"REPO_12" : "SLE-12-SP2-HA-GEO-POOL-s390x-x86_64-Build0055-Media1",
"REPO_13" : "SLE-12-SP2-HA-GEO-POOL-s390x-x86_64-Build0055-Media1.license",
"REPO_14" : "SLE-12-SP2-WE-POOL-x86_64-Build0154-Media1",
"REPO_15" : "SLE-12-SP2-WE-POOL-x86_64-Build0154-Media1.license",
"REPO_7" : "SLE-12-SP2-Server-POOL-x86_64-Build1294-Media1",
"REPO_8" : "SLE-12-SP2-SDK-POOL-x86_64-Build0542-Media1",
"REPO_9" : "SLE-12-SP2-SDK-POOL-x86_64-Build0542-Media1.license",
"SCC_CERT" : null,
"SCC_EMAIL" : "openqa@suse.de",
"SCC_REGCODE" : "SLE-12-SP2-Server-DVD-x86_64-6gXVQP2tcAcEq3iD",
"SCC_URL" : "http://scc.openqa.suse.de",
"SHUTDOWN_NEEDS_AUTH" : "1",
"SLENKINS_TESTSUITES_REPO" : "http://download.suse.de/ibs/Devel:/SLEnkins:/testsuites/SLE_12_SP2/",
"TEST" : "migration_offline_sle12sp1-update",
"UPGRADE" : "1",
"VERSION" : "12-SP2",
"VNC" : "91"
}

-> the test fail, when contacting the scc server, even in the skip registration.

Solution/Workaround:¶

1) Add manually the repos from job-maintance test. (new-test for that). this isn't a registration but the repos can be fetched for update.
-> Not nice solution, because will expose the sles-repos on github, and is an hardcoded one.

2) .. ?

@dgutu, mitiao: Please update this ticket to the current status. If I am not mistaken SLE11SP4 migration is still missing except for offline migration in the acceptance test group

online migration tests already covered fully patch and minimal necessary patches before migration.
'zypper patch --updatestack-only' would be executed by yast2/zypper migration if the system was not fully patched.

fully patch before migration
https://openqa.suse.de/tests/498586#step/zypper_patch/1

minimal patch before migration
zypper: https://openqa.suse.de/tests/498083#step/zypper_migration/1
yast : https://openqa.suse.de/tests/497749#step/yast2_migration/12

Thank you @mitiao for you comment, that explains most of the requirements for online migration.
I will focus on /offline/ one and what I've found until now is that:

1. On SLES-12-SP0 zypper should be upgraded to be able to use "zypper patch --updatestack-only" if I add DVD1 with SP2 then I get packages conflict. I can use workaround magic.

dgutu wrote:

if I add DVD1 with SP2 then I get packages conflict. I can use workaround magic.

why should you add this DVD? I guess the idea is to boot into the system, call the update command and then restart into the offline migration which will do the upgrade to SP2.

I need to patch before upgrade, that's the requirements in this task. I can't use zypper patch with option "--updatestack-only" in SLES12SP0
and I consider the source of offline-migration DVD with SLES12SP2. I'm sleepy?

okurz wrote:

dgutu wrote:

if I add DVD1 with SP2 then I get packages conflict. I can use workaround magic.

why should you add this DVD? I guess the idea is to boot into the system, call the update command and then restart into the offline migration which will do the upgrade to SP2.

I need to patch before upgrade, that's the requirements in this task. I can't use zypper patch with option "--updatestack-only" in SLES12SP0
and I consider the source of offline-migration DVD with SLES12SP2. I'm sleepy?

Or let me explain that I'm not reinventing the tests :), I'm re-using the existing offline migration test which works in Acceptance Group.

I guess it would be easier to boot into the existing test, update and reboot into offline migration test, also, reusing existing tests and scenarios, just with an additional step for this additional scenario. We want to preserve the existing scenarios anyway, not overwrite them.

okurz wrote:

I guess it would be easier to boot into the existing test, update and reboot into offline migration test, also, reusing existing tests and scenarios, just with an additional step for this additional scenario. We want to preserve the existing scenarios anyway, not overwrite them.

So you mean the order could be this:

1. Register system during installation (SLES-12-SP0 and SLES-12-SP1)
2. Install all updates/patches
3. Save/publish HDD
4. Start offline upgrade from

Am I right?

dgutu wrote:

So you mean the order could be this:

1. Register system during installation (SLES-12-SP0 and SLES-12-SP1)
2. Install all updates/patches
3. Save/publish HDD
4. Start offline upgrade from

Am I right?

No, because a registration includes a database entry inside SCC for the 12-SP0 or 12-SP1 machine

As soon as that ONE installation is upgraded, it becomes an SP2 database entry and cannot be re-used as an 12-SP0 or 12-SP1 system

That would make the reuse of the image impossible

Therefore the only option is to

1. Install without registration 12-SP0 or 12-SP1
2. Reuse the image in lots of different scenarios, boot the reused image
3. Register the 12-SP0 or 12-SP1 machine
4. Patch it
5. Reboot into 12-SP2 media and start offline/online migration

RBrownSUSE wrote:

dgutu wrote:

So you mean the order could be this:

1. Register system during installation (SLES-12-SP0 and SLES-12-SP1)
2. Install all updates/patches
3. Save/publish HDD
4. Start offline upgrade from

Am I right?

No, because a registration includes a database entry inside SCC for the 12-SP0 or 12-SP1 machine

As soon as that ONE installation is upgraded, it becomes an SP2 database entry and cannot be re-used as an 12-SP0 or 12-SP1 system

That would make the reuse of the image impossible

Therefore the only option is to

1. Install without registration 12-SP0 or 12-SP1
2. Reuse the image in lots of different scenarios, boot the reused image
3. Register the 12-SP0 or 12-SP1 machine
4. Patch it
5. Reboot into 12-SP2 media and start offline/online migration

I have reused and modified some tests which are used in migration online.
So steps are the follows:

1. Boot clear/unregistered HDD image with sles12sp0 & sles12sp1
2. Register them against Proxy-SCC
3. Get patches
4. De-register
5. Generate HDD-patched
6. Use them for offline migrations. Some examples:

http://crocodile.qa.suse.cz/tests/3990
http://crocodile.qa.suse.cz/tests/3989

Of course this will like one function :) but as a draft I think it looks good.
What about SLE11-SP4 ? The same logic?

SLE11-SP4, same logic should work for offline migrations, yes

will not work for online migrations as 'step 7' of the online migration is "boot and register" before starting the upgrade, and it takes 48 hours+ after registering a SLE 11 machine before it appears in SCC, therefore any SCC-bound tests, such as online migration will fail.

But the online migrators already know this - just stating it here for completeness, and it helps explain why testing this properly in offline migration is so important :)

What is the current status here?

maritawerner wrote:

What is the current status here?

The status is the following:

1. On the server ain.arch.suse.de I have created the test cases for all migration scenarios that we currently have on main openqa
2. I need at least a couple of days to run the tests against SP3

Some examples to check, sles11-sp4 migration (offline) to sles12-sp3
http://ain.arch.suse.de/tests/1055 - patching
http://ain.arch.suse.de/tests/1055 - migration

I still have 2 issue to resolve before creating a pull-request, this are:

1. The zypper_lr gets confused, i declare that I need to register but in fact I just need to use the update channels last run on autoupgrade_sles11sp4 - http://corr.arch.suse.de/tests/659
2. The testcase migration_zdup_offline_sle12sp2 fails to mount the installation media last run - http://corr.arch.suse.de/tests/660#step/zdup/14 This happens only in 'my' environment so o.s.d is just on this step.

dgutu wrote:

I still have 2 issue to resolve before creating a pull-request, this are:

1. The zypper_lr gets confused, i declare that I need to register but in fact I just need to use the update channels last run on autoupgrade_sles11sp4 - http://corr.arch.suse.de/tests/659

• This one was resolved by unset the SCC_REGISTRATION variable after patching

1. The testcase migration_zdup_offline_sle12sp2 fails to mount the installation media last run - http://corr.arch.suse.de/tests/660#step/zdup/14 This happens only in 'my' environment so o.s.d is just on this step.

• Today I've tried newer qemu bios, o.s.d uses much newer bios version then on mine so I've tried also newest one but without luck. Giving a try with manual testing.

dgutu wrote:

1. The testcase migration_zdup_offline_sle12sp2 fails to mount the installation media last run - http://corr.arch.suse.de/tests/660#step/zdup/14 This happens only in 'my' environment so o.s.d is just on this step.

• Today I've tried newer qemu bios, o.s.d uses much newer bios version then on mine so I've tried also newest one but without luck. Giving a try with manual testing.

• No such issue on manual testing, I'm creating the PR.

please reference PR and update ticket at least once a week. That should not be to hard for a ticket that is considered "urgent".

PR - https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/2367
WIP on suggested changes

1. I've faced a situation when one test in migration needs to registration against SCC so right now I'm adapting the patching test to not set registration variable if the test mentioned is not in place.
2. Also because of registration variable zypper_lr test fails, so I'm working on this part also.

How are you going to patch WITHOUT registration to SCC?

I think all of the scenarios probably need registration, because all the supported scenarios need patching, and patching needs registration to provide the repositories

or am I misunderstanding that dependency chain?

RBrownSUSE wrote:

How are you going to patch WITHOUT registration to SCC?

I think all of the scenarios probably need registration, because all the supported scenarios need patching, and patching needs registration to provide the repositories

or am I misunderstanding that dependency chain?

Yes, a bit of misunderstanding :) Could be my fault.
I do use the scc registration while patching the hdd and then I need to unset the registration variable, otherwise it will
try to register the system during upgrade. There is a testcase migration_offline_sle12sp1_smt which need the registration even on the step
of upgrading. Hope that explains.

failed at https://openqa.suse.de/tests/846489#step/patch_before_migration/45
looks like it is a issue about registration with addons in patch_before_migration.pm

created a ticket to solve it:
https://progress.opensuse.org/issues/18224

I agree. Keep in mind that dgutu is in holiday for the next days so someone
else might want to take over