action #23406
closedcoordination #20944: [sle][functional][epic] Get rid of architecture specific test suites
[sle][functional]Use single test suite for create_hdd_gnome on all architectures (and downstream jobs)
0%
Description
acceptance criteria¶
- AC1: There should be only one testsuite "create_hdd_gnome", not variants with a architecture specific suffix
- AC2: There should also be only one "extra_tests_gnome" because the duplication with the architecture specific suffix was only necessary because of differing "create_hdd_gnome"
tasks¶
- Review why we need a bigger HDD size for the ppc64le variant. I think it was made special for ppc6le because of the snapper_cleanup test module added by dgutu which does not seem to be that stable anyway. (0.2-4h)
- If the big HDD size for ppc64le is necessary, apply the same settings special for ppc64le to the other archs if it does not harm, e.g. if it's just about a bigger HDD size (2h-4h)
- Alternative: Keep the special ppc64le job with a good test suite description. Just specify "START_AFTER_TEST=create_hdd_textmode,create_hdd_textmode_ppc" in "extra_tests_filesystem" so that at least we don't duplicate the extra tests testsuite (0.5-1h)
further details¶
At the moment we have create_hdd_gnome_s390x and create_hdd_gnome
The only difference is in settings is that create_hdd_gnome has TOGGLEHOME=1. If this setting doesn't work for s390x, we should just change test behavior.
Updated by okurz over 6 years ago
- Subject changed from Use single test suite for create_hdd_gnome on all architectures to [sle][functional]Use single test suite for create_hdd_gnome on all architectures
Updated by mgriessmeier over 6 years ago
- Status changed from New to In Progress
apparently create_hdd_gnome_s390x was never executed (or someone deleted the results)
I've added both testsuites create_hdd_gnome and create_hdd_gnome_s390x to test development job group to see how they behave
Additionally I'm trying to reproduce it manually
Updated by mgriessmeier over 6 years ago
- Blocked by action #25638: [sles][functional][s390x] test fails in shutdown: VNC stall detected, needs to be investigated added
Updated by mgriessmeier over 6 years ago
- Status changed from In Progress to Feedback
so what I can say, the TOGGLEHOME=1 seems to have no influence on the testsuite, it looks like the same:
without TOGGLEHOME=1: http://opeth.suse.de/tests/5649
with TOGGLEHOME=1: http://opeth.suse.de/tests/5648
also
https://openqa.suse.de/tests/overview?distri=sle&version=15&build=278.1&groupid=96&failed_modules=shutdown
shows no changes in the installation workflow between the two testsuites
I've created ticket for this shutdown issue:
https://progress.opensuse.org/issues/25638
Nevertheless, I will remove the s390 specific testsuite and add s390-machine to the generic one
-> setting to feedback until blocker ticket is handled
Updated by okurz over 6 years ago
nice. Only point I don't understand how #25638 is blocking this one. If the shutdown does not work that shouldn't hinder to use the same testsuite. We can just handle the still not working case easily.
So when you removed the _s390x testsuite and added the schedule for the generic one on s390x just close the ticket I'd say.
Updated by okurz over 6 years ago
- Status changed from Feedback to In Progress
- Assignee deleted (
mgriessmeier)
I fail to find any extra tests on s390x now. Could it be mgriessmeier deleted the s390x specific variants but never added again the generic ones? What's going on? I added extra_tests_in_textmode and extra_tests_on_gnome for s390x to the test schedule for Test Development: SLE 15. I realized that "extra_test_filesystem" is misnamed so I corrected the name to follow the convention of the others on osd and o3, "extra_test_filesystem" -> "extra_tests_filesystem".
We still have a ppc64le specific job though. I see the following options:
- apply the same settings special for ppc64le to the other archs if it does not harm, e.g. if it's just about a bigger HDD size. I think it was made special for ppc6le because of the snapper_cleanup test module added by dgutu which does not seem to be that stable anyway
- just specify "START_AFTER_TEST=create_hdd_textmode,create_hdd_textmode_ppc" in "extra_tests_filesystem" so that at least we don't duplicate the extra tests testsuite
- crosscheck if ppc64le needs the special setting in "create_textmode_ppc64le" anymore and if not remove
We thought we would have been done here but we are not at all
Updated by okurz over 6 years ago
- Due date changed from 2017-10-11 to 2017-10-25
- Assignee set to riafarov
- Priority changed from Normal to High
As we have extra tests that rely on this we should work on it during the next sprint.
Updated by riafarov over 6 years ago
Blocked by https://progress.opensuse.org/issues/25638 as on x-kvm shutdown fails and we are not able to produce image, whereas there are no other reasons why we cannot reuse existing test suite for s390x.
Updated by riafarov over 6 years ago
- Status changed from In Progress to Feedback
Updated by okurz over 6 years ago
- Status changed from Feedback to In Progress
you are not blocked when the same still applies for ppc64le, see #23406#note-10
Please update ticket description or unassign.
Updated by riafarov over 6 years ago
- Description updated (diff)
- Assignee deleted (
riafarov)
Updated by okurz over 6 years ago
- Subject changed from [sle][functional]Use single test suite for create_hdd_gnome on all architectures to [sle][functional]Use single test suite for create_hdd_gnome on all architectures (and downstream jobs)
- Description updated (diff)
Updated by riafarov over 6 years ago
- Description updated (diff)
- Status changed from In Progress to Resolved
- Assignee set to riafarov
Using single test suite now. Test fails on s390x due to the issue in shutdown module, which appears in other jobs as well so not blocking this ticket. Both ACs done.