coordination #9572
closed[sles][functional][autoyast][yast][y][epic]AutoYaST testing - build autoyast.xml and then test
100%
Description
Use Upload_asset to do an install, export the autoyast, and then reinstall (on the same worker for s390x) for all builds
Suggestion: Validate profile before using for installation:
- /usr/bin/xmllint --noout --relaxng /usr/share/YaST2/schema/autoyast/rng/profile.rng data/autoinst.xml
- or validate by jing as suggested in https://trello.com/c/NBIgSFLE/402-3-create-an-autoyast-profiles-repository-to-check
Updated by RBrownSUSE over 9 years ago
- Status changed from New to In Progress
- Assignee set to RBrownSUSE
Updated by okurz over 9 years ago
different test cases¶
- autoinst.xml files generated by "Clone System Configuration" yields a file which "looks sane"
- same as above but file generated by "Export Configuration"
- same as above but file generated by "yast clone_system"
- one (or multiple) of above files can be used to install systems over different ways
- "interactive"
- serial
- ssh+no_X
- ssh+X (see https://bugzilla.suse.com/show_bug.cgi?id=955260)
- vnc
further notes¶
"looks sane" could mean that it includes sane XML, does not differ significantly vs. a "golden sample", i.e. a profile that was usable before for installation or one profile file vs. the others don't differ significantly.
Proposed to use xmllint --format --exc-c14n to sanitize XML first (see http://superuser.com/questions/79920/how-can-i-diff-two-xml-files#comment1114741_459008), then compare with plain text diff, ignore known entries that change, e.g. password hash, uuid of root, etc., assert rest is equal.
Also see:
Updated by RBrownSUSE over 9 years ago
- Status changed from In Progress to New
- Assignee deleted (
RBrownSUSE) - Priority changed from Urgent to High
Updated by cwh over 9 years ago
I actually implemented a related test for validating bsc#956012 - Default route not present in profile cloned at the end of installation.
I am - for now - just using 'grep' to check the element's existence. Using xmllint is definitly the better option.
What I already did is IMO a good base to use xmllint later. Find pull request here:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/891
Note that this test only succeeds when using a static network config. Therefore it needs
EXTRABOOTPARAMS="ifcfg='eth0=10.0.2.22/24,10.0.2.2,10.0.2.3,susetest'"
(and AY_PROFILECHECK=1 to be actually schedueled)
to succeed.
Updated by okurz over 9 years ago
what was already done looks good. What else is necessary to be done should be discussed keeping in mind what YaST team already tests. for references see
https://etherpad.nue.suse.com/p/autoyast-tests-on-openqa
and
https://github.com/yast/autoyast-integration-test/pull/75
Updated by RBrownSUSE over 9 years ago
- Checklist item changed from to [ ] SLE, [ ] TW, [ ] Leap
- Target version deleted (
154)
Updated by mkravec over 8 years ago
- Status changed from New to In Progress
- % Done changed from 0 to 70
Working locally for SLES, PR ready for osd - https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/2163
Updated by okurz over 8 years ago
PR merged, do you need to add scenarios? Please reference successful runs from osd then.
Updated by mkravec over 8 years ago
Scenarios added for SLES & TW, waiting for next build
Updated by mkravec over 8 years ago
- Status changed from In Progress to Feedback
Scenarios are scheduled:
SLES - https://openqa.suse.de/tests/692575 (aarch64,ppc64le,x86_64)
LEAP - https://openqa.opensuse.org/tests/336185
TW - bsc#1017490 - https://openqa.opensuse.org/tests/329554#step/yast2_clone_system/3
s390x test is still on todo list
Updated by mgriessmeier over 8 years ago
- Has duplicate action #11922: [sle][functional][s390][yast][y][hard] Add AutoYaST tests for s390 added
Updated by mkravec over 8 years ago
Currently failing on bsc#1013605 on s390 - waiting for bugfix or workaround (mgriessmeier is working on it)
https://openqa.suse.de/tests/732316#step/yast2_clone_system/3
Updated by mgriessmeier about 8 years ago
created PR with workaround needle: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/2424
Updated by mkravec about 8 years ago
- Status changed from Feedback to In Progress
bootloader_zkvm ignores AUTOYAST variable, needs to be modified to type it's content
Updated by mgriessmeier about 8 years ago
mkravec wrote:
bootloader_zkvm ignores AUTOYAST variable, needs to be modified to type it's content
PR: https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/2479
Updated by mkravec about 8 years ago
We need to find a way to detect reboot after autoyast stage1 and reconnect to continue with stage2
Updated by okurz about 8 years ago
- Target version changed from Milestone 5 to Milestone 6
Updated by mkravec about 8 years ago
It might be possible to use "assert_screen black" to detect reboot after first stage - will try
Updated by okurz about 8 years ago
- Subject changed from AutoYaST testing - build autoyast.xml and then test to [sles][functional]AutoYaST testing - build autoyast.xml and then test
Updated by okurz about 8 years ago
- Target version changed from Milestone 6 to Milestone 7
@mkravec: please write an update from your side
I guess mkravec-ay-generate-s390 and mkravec-ay-reinstall-s390 are the corresponding development jobs which need to be fixed before we can close this ticket. The first one looks fine but the second one stops currently in https://openqa.suse.de/tests/878380#step/installation/2 which could be related to "download problems" after caching implementation. Can you please take a closer look?
Updated by okurz almost 8 years ago
- Target version changed from Milestone 7 to Milestone 8
Updated by mkravec almost 8 years ago
- Assignee deleted (
mkravec)
Test needs to detect reboot and reconnect after autoyast first stage finishes on s390. We didn't find good way to do this.
I will unassign myself for now and will continue when I have more time.
Updated by okurz almost 8 years ago
- Priority changed from High to Normal
ok. As we have most of the parts covered already and it seems only s390x SLE is missing prio should be lower priority. High -> Normal
Updated by sebchlad over 7 years ago
- Target version set to future
Moving to the target version: "future" so we could add this to milestones while planning upcoming milestones.
Updated by riafarov over 7 years ago
- Status changed from In Progress to Workable
FOr SLE is done for all architectures except s390x
Updated by okurz about 7 years ago
- Has duplicate deleted (action #11922: [sle][functional][s390][yast][y][hard] Add AutoYaST tests for s390)
Updated by okurz about 7 years ago
- Subject changed from [sles][functional]AutoYaST testing - build autoyast.xml and then test to [sles][functional][autoyast][yast][y][epic]AutoYaST testing - build autoyast.xml and then test
- Status changed from Workable to Blocked
- Assignee set to riafarov
- Target version changed from future to Milestone 16
@riafarov the (now) subtask for s390x autoyast installations as described in #11922 seem to be way more feasible since @mitiao seems to do the major work regarding enabling s390x autoyast installations with https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/4561 . Therefore I think we should make use of that recent drive, make sure we have s390x covered in autoyast installations - in #11922 - and then think about further necessary s390x autoyast tests that we want to cover as new tickets. This is why I am assigning this ticket to you to lead the thinking process about which new tests to add on s390x autoyast.
Updated by riafarov about 7 years ago
- Assignee deleted (
riafarov)
It's new tests, do not see it as priority. Solution I saw was not perfect and we need more time to make it right. I proposed more thing which are easier to reach. So let's discuss during milestone 16. I'll un-assign myself as it's blocked (not sure by what though).
Updated by riafarov about 7 years ago
- Due date changed from 2018-04-10 to 2018-04-24
due to changes in a related task
Updated by okurz about 7 years ago
- Target version changed from Milestone 16 to Milestone 17
Updated by mgriessmeier about 7 years ago
cannot edit due date, will be shifted once we've scheduled the subticket
Updated by riafarov about 7 years ago
- Due date changed from 2018-04-24 to 2018-07-18
due to changes in a related task
Updated by okurz almost 7 years ago
- Due date changed from 2018-07-18 to 2018-07-17
due to changes in a related task
Updated by okurz almost 7 years ago
- Target version changed from Milestone 17 to Milestone 17
Updated by riafarov almost 7 years ago
- Due date changed from 2018-07-17 to 2018-07-03
due to changes in a related task
Updated by riafarov almost 7 years ago
- Status changed from Workable to Resolved
- Assignee set to riafarov
Scenario enabled on x kvm
Updated by szarate over 4 years ago
- Tracker changed from action to coordination
Updated by szarate over 4 years ago
See for the reason of tracker change: http://mailman.suse.de/mailman/private/qa-sle/2020-October/002722.html