action #56585
closed[qe-yast][qam][functional][sporadic] test fails in yast2_lan_device_settings with approximate error rate of 70% looking for static ip but dhcp is shown
20%
Description
Observation¶
openQA test in scenario sle-12-SP5-Server-DVD-aarch64-poo54128_extra_tests_on_gnome_002@aarch64 fails in
yast2_lan_device_settings
with approximate error rate of 70% looking for static ip but dhcp is shown.
The test code tries to select a static ip adress but fails to do so as also visible in the video. It seems there are too many actions without a sync in between, e.g. no assert_screen or other confirmation of the selected actions.
Reproducible¶
Seems to fail in 70% of all cases at least on openqaworker-arm-3
Expected result¶
Should be able to run stable
Further details¶
Always latest result in this scenario: latest
Updated by vsvecova over 4 years ago
Thanks for the report, Oliver, I'll look into it
Updated by riafarov over 4 years ago
- Subject changed from [qam][functional][y][sporadic] test fails in yast2_lan_device_settings with approximate error rate of 70% looking for static ip but dhcp is shown to [qam][functional][sporadic] test fails in yast2_lan_device_settings with approximate error rate of 70% looking for static ip but dhcp is shown
As Veronika works on it, I remove [y] tag. Cheers.
Updated by SLindoMansilla over 4 years ago
- Has duplicate action #58808: test fails in yast2_lan_device_settings added
Updated by okurz over 4 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: extra_tests_on_gnome
https://openqa.suse.de/tests/3575889
To prevent further reminder comments one of the following options should be followed:
- The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
- The openQA job group is moved to "Released"
- The label in the openQA scenario is removed
Updated by okurz over 4 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: extra_tests_on_gnome
https://openqa.suse.de/tests/3575889
To prevent further reminder comments one of the following options should be followed:
- The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
- The openQA job group is moved to "Released"
- The label in the openQA scenario is removed
Updated by okurz over 4 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: extra_tests_in_textmode@svirt-xen-pv
https://openqa.suse.de/tests/3629386
To prevent further reminder comments one of the following options should be followed:
- The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
- The openQA job group is moved to "Released"
- The label in the openQA scenario is removed
Updated by okurz over 4 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: extra_tests_in_textmode@svirt-xen-pv
https://openqa.suse.de/tests/3658854
To prevent further reminder comments one of the following options should be followed:
- The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
- The openQA job group is moved to "Released"
- The label in the openQA scenario is removed
Updated by okurz over 4 years ago
This is an autogenerated message for openQA integration by the openqa_review script:
This bug is still referenced in a failing openQA test: extra_tests_in_textmode@svirt-xen-pv
https://openqa.suse.de/tests/3658854
To prevent further reminder comments one of the following options should be followed:
- The test scenario is fixed by applying the bug fix to the tested product or the test is adjusted
- The openQA job group is moved to "Released"
- The label in the openQA scenario is removed
Updated by tjyrinki_suse about 4 years ago
- Assignee deleted (
vsvecova)
Unassigning since not in progress, please correct if wrong.
Updated by tjyrinki_suse almost 4 years ago
- Status changed from New to Workable
Updated by mgrifalconi over 3 years ago
I tried to schedule a few test runs:
https://openqa.suse.de/tests/4568823
https://openqa.suse.de/tests/4568839
https://openqa.suse.de/tests/4568843
https://openqa.suse.de/tests/4568888
https://openqa.suse.de/tests/4568909
https://openqa.suse.de/tests/4568910
And they are all green so far
Full test cloned, with no special schedule https://openqa.suse.de/tests/4568915
Updated by mgrifalconi over 3 years ago
Scheduling one more run of the latest job with no exclusion of yast2_lan_device_settings and the module worked fine.
https://openqa.suse.de/tests/4694387
@okurz would it make sense to re-enable the test and see if the issue is gone now? Otherwise can you suggest a way to reproduce the problem?
Thank you!
Updated by okurz over 3 years ago
The problem was that the problem happened in 70% of the cases so a single job run does not show us if it's stable. I suggest you follow the proposal in https://progress.opensuse.org/projects/openqatests/wiki/Wiki#Statistical-investigation to check for the stability with multiple test runs. If the problem does not happen within a sufficient number of tests then you can of course re-enable the test again as part of the usual test schedule
Updated by mgrifalconi over 3 years ago
- % Done changed from 0 to 20
Scheduled 20 cases to start: https://openqa.suse.de/tests/overview?build=poo56585_investigation
Apart few timeouts where it did not start, "yast2_lan_device_settings" was always green.
"vino_screensharing_available" was consistently failing by the way, but not in scope of this poo.
Will schedule some more runs.
Updated by mgrifalconi over 3 years ago
Hello @okurz,
I ran a few more tests: https://openqa.suse.de/tests/overview?build=poo56585_investigation
Apart from some runs that failed to start due to an old job being cloned (assets were deleted already), I see all jobs that run only yast2_lan_device_settings
successful and only 3 runs with the full job where yast2_lan_device_settings failed:
https://openqa.suse.de/tests/4716042#
https://openqa.suse.de/tests/4716064#
https://openqa.suse.de/tests/4716049#
But it looks to me a different issue, since in all cases the IP is already there.
If I compare the screenshot of a successful job with the step where it fails, it looks the same to me, but the needle does not get 100%.
Successful: https://openqa.suse.de/tests/4716060#
Am I missing something obvious?
Thanks,
Michael
Updated by tjyrinki_suse over 3 years ago
- Subject changed from [qam][functional][sporadic] test fails in yast2_lan_device_settings with approximate error rate of 70% looking for static ip but dhcp is shown to [qe-core][qam][functional][sporadic] test fails in yast2_lan_device_settings with approximate error rate of 70% looking for static ip but dhcp is shown
Updated by tjyrinki_suse over 3 years ago
- Project changed from openQA Tests to qe-yam
- Subject changed from [qe-core][qam][functional][sporadic] test fails in yast2_lan_device_settings with approximate error rate of 70% looking for static ip but dhcp is shown to [qe-yast][qam][functional][sporadic] test fails in yast2_lan_device_settings with approximate error rate of 70% looking for static ip but dhcp is shown
- Category deleted (
Bugs in existing tests)
Updated by riafarov over 3 years ago
- Project changed from qe-yam to openQA Tests
@tjyrinki_suse I see you have assigned project to qe-yast. But ticket is quite old, and as per last comment from Michael, I would propose to reject it.
Updated by tjyrinki_suse over 3 years ago
- Status changed from Workable to Rejected
Sounds like it might be gone indeed now from that analysis.