action #39575
closed[functional][u] Split consoletest_setup to smaller parts which serve single purpose per module
Added by riafarov over 5 years ago. Updated over 5 years ago.
0%
Description
Motivation¶
consoletest_setup became a place to put too many things which are coming from different area. With PR#5564 we tried to introduce split by criteria if change is permanent and should stay in the SUT, or should be performed before test run but should not remain (e.g. reverted).
One outcome we found was the issue with installing tar and curl, so we had to stop packagekit service upfront.
Some feedback from Josef:
"For me this looks like system prepare could be scheduled only for tests with PUBLISH_HDD_*
Grub part from should be in grub_set_bootargs
ensure_serialdev_permissions and curl tar install would need to be also in consoletest_setup"
Acceptance criteria¶
- consoletest_setup/system_prepare test modules are split to more granular parts to serve single purpose per module
Suggestions¶
We need to consider all scenarios before introducing these changes, x11 tests, maintenance, and Functional tests.
Updated by okurz over 5 years ago
- Subject changed from [functional] Split consoletest_setup to smaller parts which serve single purpose per module to [functional][u] Split consoletest_setup to smaller parts which serve single purpose per module
- Target version set to Milestone 19
Updated by okurz over 5 years ago
- Due date set to 2018-11-06
- Target version changed from Milestone 19 to Milestone 20
We have improved recently but I still wonder if we can not simplify a bit more, e.g. https://openqa.suse.de/tests/2105865/file/autoinst-log.txt are the logs from a yast2_ncurses test which run consoletest_setup which takes 2 minutes to execute. https://openqa.suse.de/tests/2094825 is "extra_tests_on_gnome" which also executes consoletest_setup and I wonder if we can spare some steps there still.
Updated by zluo over 5 years ago
- Status changed from New to Workable
- Assignee set to zluo
take over...
Updated by zluo over 5 years ago
after I discussed with mgriessmeier, we need to check consoletest_setup at first for function split.
Updated by zluo over 5 years ago
checking for extra_tests_on_gnome@64bit:
http://e13.suse.de/tests/9717
check_test_data, uninstall_pattern and check_network added for testing.
Updated by zluo over 5 years ago
extra_tests_in_textmode@64bit, yast2_ncurses@64bit, extra_tests_filesystem@64bit are fine for the split:
http://e13.suse.de/tests/9728
http://e13.suse.de/tests/9727
http://e13.suse.de/tests/9725
Updated by zluo over 5 years ago
clone_system@64bit:
http://e13.suse.de/tests/9729
Updated by okurz over 5 years ago
looks interesting, can you provide a WIP-PR so that we see how it would look like from code-perspective?
http://e13.suse.de/tests/9730#step/textinfo/4 fails because of your change because you extracted the test data as root in http://e13.suse.de/tests/9730#step/check_test_data/9 . Also it seems we need to address the original goal which is maybe hard to readout from the description but I recommend the following: Check what parts need to be actually scheduled e.g. for "extra_tests_in_textmode" as the parent job should already schedule parts of system preparation. Same for "gnome". The goal is to save time in job execution not needing to execute unnecessary test module.
Updated by zluo over 5 years ago
http://e13.suse.de/tests/9737 textinfo works now :)
user-console used now in check_test_data.
Updated by zluo over 5 years ago
http://e13.suse.de/tests/9747 shows successful test run without new test modules:
check_network
uninstall_pattern
curl_upload_logs
check_test_data
Updated by zluo over 5 years ago
Updated by zluo over 5 years ago
http://e13.suse.de/tests/9750 shows successful test run for extra_tests_in_textmode without check_network,uninstall_pattern,curl_upload_logs
check_test_data is required.
Updated by zluo over 5 years ago
http://e13.suse.de/tests/9753 shows successful test run for extra_tests_filesystem:
with new consoletest_setup only, other tests are not required.
Updated by zluo over 5 years ago
extra_tests_in_textmode (TW) shows successful test runs:
http://e13.suse.de/tests/9760
extra_tests_filesystem (TW) shows successufl test runs:
http://e13.suse.de/tests/9761
Updated by zluo over 5 years ago
extra_tests_in_textmode (Leap)
http://e13.suse.de/tests/9763 shows successful test run
extra_tests_filesystem (Leap)
http://e13.suse.de/tests/9762 shows successful test run
Updated by zluo over 5 years ago
opensuse-15.1-DVD-x86_64-Build322.2-extra_tests_on_gnome@64bit shows problem on my local worker:
http://e13.suse.de/tests/9789#step/consoletest_setup/9
for systemctl "mask serial-getty\@$testapi::serialdev" if check_var('BACKEND', 'qemu');
got write error: Input/output error
and it works fine on 3o:
https://openqa.opensuse.org/tests/781724
Updated by okurz over 5 years ago
cool, that's a problem that I was looking into with riafarov. https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6027 might help. Please make sure to have that changes included in your local tests.
Updated by zluo over 5 years ago
my local branch got updated, but I still have problem with consoletest_setup, for SLES and Leap:
Updated by zluo over 5 years ago
extra_tests_on_gnome:
http://e13.suse.de/tests/9808 shows now finally successful test run after https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6061 got merged
Updated by okurz over 5 years ago
- Related to action #42359: [functional][y][sporadic] test fails in gnuhealth_install to write to /dev/ttyS0 "Input/output error" while serial getty seems to restart at the same time? added
Updated by zluo over 5 years ago
PR got updated now:
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/6032/files
Updated by zluo over 5 years ago
- Status changed from Feedback to In Progress
need to update PR for now.
Updated by okurz over 5 years ago
- Target version changed from Milestone 20 to Milestone 21
Updated by zluo over 5 years ago
This breaks argon and krypton tests of opensuse:
https://openqa.opensuse.org/tests/811474
https://openqa.opensuse.org/tests/811476
Updated by zluo over 5 years ago
Updated by okurz over 5 years ago
- Status changed from In Progress to Resolved
PR merged, something else planned? I guess we are good now as also Argon is green again: https://openqa.opensuse.org/tests/812940