QA (public) » openQA Project (public) » openQA Tests (public) » qe-yam

https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/3206

PR merged, test module does no harm anymore for the yast2_ui scenario. Created new test suite "yast2_ui_devel" triggered after yast2_ui and added to development job groups for openSUSE Tumbleweed as well as openSUSE Leap.

Should check how the test module behaves for SLE

SLE tests fails in http://lord.arch/tests/latest#step/yast2_cmdline/6 with the source repo not being found. Would need to add iso2

foo

foo

foo

foo

foo

Issue with hdd_1 variable.

[2018-05-31T04:55:48.0972 CEST] [debug] CACHE: Download of /var/lib/openqa/cache/SLES-15-x86_64-657.1-gnome@64bit-gnome.qcow2 failed with: 404 - Not Found
[2018-05-31T04:55:48.0973 CEST] [debug] CACHE: removed /var/lib/openqa/cache/SLES-15-x86_64-657.1-gnome@64bit-gnome.qcow2

qcow2 image is missing "Module development tools", hence we are getting missing package error.

Possible fix using SUSEConnect -p sle-module-development-tools/15/x86_64

Successful run:

http://dhcp151.suse.cz/tests/3380#step/yast2_cmdline/16

Tested modules:

• yast2-network
• yast2-http-server

OpenQA variables:

• DESKTOP=textmode
• HDD_1=sle-15-x86_64-661.1@64bit-minimal_with_sdk661.1_installed.qcow2
• Y2UITEST_DEVEl=1
• Y2UITEST_NCURSES=1

PR to make it work on sle
https://github.com/os-autoinst/os-autoinst-distri-opensuse/pull/5171

pr merged

test suite yast2_ui_devel added to sle12sp4's functional group.

Once we have run on osd, please, set as blocked with https://progress.opensuse.org/issues/34732

mloviska wrote:

test suite yast2_ui_devel added to sle12sp4's functional group.

I think in this case it is better to track the scenario in "test development". We should not add scenarios that never worked and we don't know if they will ever work

This one is already working for sle15, however I haven't made it in time so we can have runs in OSD. I am afraid that test case will be forgotten by moving it to dev group. Considering that the job is currently block by missing dependency it might be actually easier to track the bug in functional group than in dev.

mloviska wrote:

I am afraid that test case will be forgotten by moving it to dev group

Yes, I share your concerns. This is why we still have this progress ticket. Based on that the ticket is still open and blocked we now that the test can not work. The job within openQA should also be labeled by this. The validation job groups in openQA should be reserved for tests that we expect to work. I know we did not apply this cleanly during the past months and review turned out to miss some failures which never went away. Btw, this has been mentioned as a concern by RMs as well so the suggestion is to apply more strictly our best practices we followed in before: Whenever we know that a test will not be fixed very soon we track it in "test development" based on progress tickets. It's easier for the progress ticket assignee to review a single job and determine if the job failure reason is still the same in contrast to the (overwhelmed) daily reviewer trying to get a hold of all failures.

Valid point, yast2_ui_devel is currently in dev group of sle12sp4.

Runs fine on sle15 sle-15-Installer-DVD-x86_64-Build665.2-yast2_ui_devel@64bit

This is an autogenerated message for openQA integration by the openqa_review script:

This bug is still referenced in a failing openQA test: yast2_ui_devel
https://openqa.suse.de/tests/2096097

In case of test case bug as this one. We can update the test suite by ourselves

PR to fix type for yast2 lan module
https://github.com/yast/yast-network/pull/686

Maybe we can decommission this module and related test suites since we test CLI only for 2 yast2 modules and QAM is working on their own way how to test CLI for yast2 modules anyway.
QAM test plan

PR: Add missing packages for yast2_ui_devel

VR: https://openqa.opensuse.org/tests/1019461

Now the package is present. Created bug: Broken yast command line tests