[functional][y][yast][easy] Add yast2_ui_devel scenario (was: please remove yast2_cmdline from 42.3 test plan)
WAS: please remove yast2_cmdline from the 42.3 test plan. It fails all the time and doesn't seem to serve a useful purpose.
We would like to run yast command line tests (see http://yastgithubio.readthedocs.io/en/latest/how-to-write-tests/) against packages.
Test module was abandoned for a while, but this part is not covered anywhere at the moment.
- yast2_cmdline test module is working or failing on a valid bug
- yast2_cmdline test module is executed in job group which is reviewed (not development job groups)
- Learn how to run tests and adjust code
- For openSUSE it's tricky as we don't sync source repo, but still can get tests from official source repo, elaborate with riafarov or okurz
- Adjust schedule to run test module as a part of extra tests in textmode or as a separate test suite
Adding extra source package to be synced should be added here: https://gitlab.suse.de/openqa/scripts/blob/master/rsync_opensuse.pm#L459
#1 Updated by okurz almost 4 years ago
- Status changed from New to In Progress
yast2_cmdline also fails consistently on openSUSE Tumbleweed. There it's marked with a bug but not regarded as high priority. On SLES currently we don't run this module (waiting for zluo here). I will try to exclude yast2_cmdline by default as it highly depends on external test packages by developers so I will split the scenarios.
#3 Updated by okurz almost 4 years ago
#4 Updated by okurz almost 4 years ago
SLE tests fails in http://lord.arch/tests/latest#step/yast2_cmdline/6 with the source repo not being found. Would need to add iso2
#11 Updated by okurz over 3 years ago
- Subject changed from please remove yast2_cmdline from 42.3 test plan to Add yast2_ui_devel scenario (was: please remove yast2_cmdline from 42.3 test plan)
Added "yast2_ui_devel" to SLE15 test plan for now in the development group. Let's gather some testing results before evaluation.
#14 Updated by okurz about 3 years ago
- Subject changed from [functional]Add yast2_ui_devel scenario (was: please remove yast2_cmdline from 42.3 test plan) to [functional][y][yast]Add yast2_ui_devel scenario (was: please remove yast2_cmdline from 42.3 test plan)
- Status changed from Feedback to New
- Assignee deleted (
Over to [y]-team
#15 Updated by okurz almost 3 years ago
- Subject changed from [functional][y][yast]Add yast2_ui_devel scenario (was: please remove yast2_cmdline from 42.3 test plan) to [functional][y][yast] Add yast2_ui_devel scenario (was: please remove yast2_cmdline from 42.3 test plan)
- Due date set to 2018-06-05
riafarov I think this is something you could clarify with the yast team
#18 Updated by mloviska almost 3 years ago
Issue with hdd_1 variable.
[2018-05-31T04:55:48.0972 CEST] [debug] CACHE: Download of /var/lib/openqa/cache/SLESemail@example.com failed with: 404 - Not Found [2018-05-31T04:55:48.0973 CEST] [debug] CACHE: removed /var/lib/openqa/cache/SLESfirstname.lastname@example.org
#20 Updated by mloviska almost 3 years ago
#24 Updated by mloviska almost 3 years ago
PR to make it work on sle
#35 Updated by mloviska almost 3 years ago
This one is already working for sle15, however I haven't made it in time so we can have runs in OSD. I am afraid that test case will be forgotten by moving it to dev group. Considering that the job is currently block by missing dependency it might be actually easier to track the bug in functional group than in dev.
#36 Updated by okurz almost 3 years ago
I am afraid that test case will be forgotten by moving it to dev group
Yes, I share your concerns. This is why we still have this progress ticket. Based on that the ticket is still open and blocked we now that the test can not work. The job within openQA should also be labeled by this. The validation job groups in openQA should be reserved for tests that we expect to work. I know we did not apply this cleanly during the past months and review turned out to miss some failures which never went away. Btw, this has been mentioned as a concern by RMs as well so the suggestion is to apply more strictly our best practices we followed in before: Whenever we know that a test will not be fixed very soon we track it in "test development" based on progress tickets. It's easier for the progress ticket assignee to review a single job and determine if the job failure reason is still the same in contrast to the (overwhelmed) daily reviewer trying to get a hold of all failures.
#39 Updated by okurz almost 3 years ago
- Due date changed from 2018-07-03 to 2018-08-28
- Target version changed from Milestone 17 to Milestone 18
Considering blocked now by the mentioned bug. I do not expect the resolution within the timeframe of the next sprint so let's take a look at the latest again in S24. Of course if the bug is resolved in before feel free to bring it up again.
#40 Updated by mloviska almost 3 years ago
Runs fine on sle15 sle-15-Installer-DVD-x86_64-Build665.2-yast2_ui_devel@64bit
#42 Updated by mloviska over 2 years ago
- Due date deleted (
I am going to be informed by mail anyway, thus it should be fine to put this one back to backlog again. Nevertheless we need to wait what is going to happen with Bug 1096302 - rspec from BSK cannot be installed first. I will remove the due date for now.
#49 Updated by riafarov over 1 year ago
- Due date set to 2019-08-27
- Status changed from Blocked to Workable
- Target version changed from Milestone 26 to Milestone 27
Not sure why it was blocked. In recent runs we miss some source packages, which should be easy to add even in case we don't sync them.
Might be that there are more issues hidden by the first one, but let's try.
- Project changed from openQA Tests to qe-yast
- Category deleted (
Bugs in existing tests)
- Status changed from Blocked to Resolved
Isn't it done now? I see https://openqa.opensuse.org/tests/1441560# being green for a months now. Feel free to reopen in case you believe I miss something.