action #42671
closed[opensuse][functional][u] test fails in vlc - needle vlc-done-playing does not match
0%
Description
Observation¶
openQA test in scenario opensuse-15.1-NET-x86_64-xfce@64bit-2G fails in
vlc
Needle vlc-done-playing does not match, because vlc window is shifted. … or because the video never started. We should have a bug for that -> boo#1102838 which is assigned to … okurz because we need to crosscheck better in tests what is happening.
Reproducible¶
Fails since (at least) Build 319.3 (current job)
Expected result¶
Last good: 319.2 (or more recent)
Acceptance criteria¶
- AC1: The VLC test module runs stable (statistics) being able to play the video or reference boo#1102838 when it cannot then closes VLC
Suggestions¶
- Read our story from #37713 when we tried to catch the situation that the pause button shows up including the duration of the video 0:23 but the video never actually starts to play. The problem is that in this situation VLC does not react on a click on the "close" icon
- Add back a needle to detect "pause button and 0:23 shown", e.g. as in https://openqa.opensuse.org/tests/835278/file/video.ogv#t=318.29,318.33
- If the above detected record_soft_failure pointing to boo#1102838 and make sure VLC can be closed, if not by click then "xkill" or similar
Further details¶
Always latest result in this scenario: latest
Files
Updated by okurz over 5 years ago
- Related to action #42674: [opensuse][functional][u][sporadic] test fails in vlc - cannot open video added
Updated by okurz over 5 years ago
- Description updated (diff)
- Status changed from New to Feedback
- Assignee set to okurz
well, the original job is gone and the latest one or predecessors do not really reveal what you have seen originally. E.g. https://openqa.opensuse.org/tests/833129#step/vlc/26 fails (again) because of the check for the completely typed URL. I don't understand why people create another bunch of needles, not cleaning up old ones but not creating new ones with lower match level. I did that now again same as I did for other window managers already. Also I deleted needles that did not match for the past 3 months anywhere over https://openqa.opensuse.org/admin/needles . Also adapted description to describe what I see in current jobs still, that the playing of the video actually never starts.
Updated by okurz over 5 years ago
- Related to action #42773: [qe-core][functional] extend VLC test to check correct rendering added
Updated by okurz over 5 years ago
- Related to action #37713: [functional][u] test fails in vlc - sometimes fails to close vlc added
Updated by okurz over 5 years ago
Updated by okurz over 5 years ago
one test was fine but did not trigger the actual workaround (I have not even created the needle to detect the bug situation).
Retriggered a custom scenario for statistical investigation:
for i in {001..001}; do openqa-clone-job --within-instance https://openqa.opensuse.org --skip-chained-deps 835553 _GROUP="Development Tumbleweed" TEST="update_Leap_42.3_gnome@okurz__os-autoinst-distri-opensuse_fix_vlc_$i" EXCLUDE_MODULES=keymap_or_locale,textinfo,hostname,xorg_vt,ncurses,yast2_lan,curl_https,salt,glibc_sanity,zypper_in,yast2_i,yast2_bootloader,vim,firewall_enabled,sshd,ssh_cleanup,sshfs,mtab,orphaned_packages_check,user_gui_login,desktop_runner,xterm,sshxterm,gnome_control_center,gnome_tweak_tool,gnome_terminal,gedit,firefox,firefox_audio,thunderbird,chromium,ooffice,oomath,oocalc,glxgears,nautilus,gnome_music,evolution,desktop_mainmenu,inkscape,gimp,hexchat,reboot_gnome,cleanup_before_shutdown,shutdown; done
-> Created job #835900: opensuse-Tumbleweed-DVD-x86_64-Buildokurz_os-autoinst-distri-opensuse_6614-update_Leap_42.3_gnome@okurz__os-autoinst-distri-opensuse_fix_vlc@64bit -> https://openqa.opensuse.org/t835900
passed. Let's trigger more
for i in {002..040}; do openqa-clone-job --within-instance https://openqa.opensuse.org --skip-chained-deps 835553 _GROUP="Development Tumbleweed" TEST="update_Leap_42.3_gnome@okurz__os-autoinst-distri-opensuse_fix_vlc_$i" EXCLUDE_MODULES=keymap_or_locale,textinfo,hostname,xorg_vt,ncurses,yast2_lan,curl_https,salt,glibc_sanity,zypper_in,yast2_i,yast2_bootloader,vim,firewall_enabled,sshd,ssh_cleanup,sshfs,mtab,orphaned_packages_check,user_gui_login,desktop_runner,xterm,sshxterm,gnome_control_center,gnome_tweak_tool,gnome_terminal,gedit,firefox,firefox_audio,thunderbird,chromium,ooffice,oomath,oocalc,glxgears,nautilus,gnome_music,evolution,desktop_mainmenu,inkscape,gimp,hexchat,reboot_gnome,cleanup_before_shutdown,shutdown; done
One failed in force_scheduled_tasks for what is another ticket, one incompleted, reported as #46556, others passed. So ignoring other failures we can see 40/40 passed meaning that I can not reproduce the error. This might be due to excluding too many modules which might have an impact so I will include them back and let the tests run again.
for i in {001..040}; do openqa-clone-job --within-instance https://openqa.opensuse.org --skip-chained-deps 835553 _GROUP="Development Tumbleweed" TEST="update_Leap_42.3_gnome@okurz__os-autoinst-distri-opensuse_fix_vlc_$i" BUILD=okurz/os-autoinst-distri-opensuse#6614_all_modules EXCLUDE_MODULES=reboot_gnome,cleanup_before_shutdown,shutdown; done
Updated by okurz over 5 years ago
this will most likely take until tomorrow – or fail fast early ;)
Updated by okurz over 5 years ago
Unexpectedly 1/40 failed. I guess I need to trigger some more to have good statistics?
waiting … (same link)
EDIT: So 3/80 failed, that is 3.8%. Not much but ok for reproducability I guess.
Updated by okurz over 5 years ago
ok, I think I will play it safe and create an explicit new needle tag and created a needle "vlc-stuck-never-played-boo1102838-20190125" for that already. Updated code, retriggering …
for i in {001..080}; do openqa-clone-job --within-instance https://openqa.opensuse.org --skip-chained-deps 835553 _GROUP="Development Tumbleweed" TEST="update_Leap_42.3_gnome@okurz__os-autoinst-distri-opensuse_fix_vlc_$i" BUILD=okurz/os-autoinst-distri-opensuse#6614_all_modules_with_workaround EXCLUDE_MODULES=reboot_gnome,cleanup_before_shutdown,shutdown; done
stupid me. This can not work because still we need to pass first the step of "done-playing". Well, let's look for the alternative explicitly but earlier then?
adapted, pushed, retriggered:
for i in {001..080}; do openqa-clone-job --within-instance https://openqa.opensuse.org --skip-chained-deps 838337 _GROUP="Development Tumbleweed" TEST="update_Leap_42.3_gnome@okurz_os-autoinst-distri-opensuse_fix_vlc_$i" BUILD=okurz/os-autoinst-distri-opensuse#6614_all_modules_with_workaround2 EXCLUDE_MODULES=reboot_gnome,cleanup_before_shutdown,shutdown; done
Made a mistake by using a "latest" job but not specifying CASEDIR again
https://github.com/okurz/os-autoinst-distri-opensuse.git#fix/vlc
for i in {001..100}; do openqa-clone-job --within-instance https://openqa.opensuse.org --skip-chained-deps 838337 _GROUP="Development Tumbleweed" TEST="update_Leap_42.3_gnome@okurz_os-autoinst-distri-opensuse_fix_vlc_$i" BUILD=okurz/os-autoinst-distri-opensuse#6614_all_modules_with_workaround2 EXCLUDE_MODULES=reboot_gnome,cleanup_before_shutdown,shutdown CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse.git#fix/vlc; done
All failed in zypper_ref because apparently the older snapshot (it's actually just HEAD~1) is gone already. Retriggering based on latest, job https://openqa.opensuse.org/tests/839882
for i in {001..100}; do openqa-clone-job --within-instance https://openqa.opensuse.org --skip-chained-deps 839882 _GROUP="Development Tumbleweed" TEST="update_Leap_42.3_gnome@okurz_os-autoinst-distri-opensuse_fix_vlc_$i" BUILD=okurz/os-autoinst-distri-opensuse#6614_all_modules_with_workaround2 EXCLUDE_MODULES=reboot_gnome,cleanup_before_shutdown,shutdown CASEDIR=https://github.com/okurz/os-autoinst-distri-opensuse.git#fix/vlc; done
based on NET now
https://openqa.opensuse.org/tests/840423#step/vlc/27
Searching over https://openqa.opensuse.org/admin/needles for the needle tag "vlc-stuck-never-played" reveals the last match https://openqa.opensuse.org/tests/840423#step/vlc/27 with the workaround of "killall -9 vlc" triggered in https://openqa.opensuse.org/tests/840423#step/vlc/31 so the workaround is effective.
Updated by okurz about 5 years ago
- Blocks action #39926: [functional][u][sporadic][virtio] test fails in vlc - vlc is started twice, needle target_match_vlc seems wrong added
Updated by okurz about 5 years ago
Updated by okurz about 5 years ago
- Status changed from Feedback to Resolved
Adressed the usual openSUSE test reviewers additionally in [#opensuse-factory[01/02/2019 09:02:19] DimStar, maxlin, Marcus, lnussel, michel_mno: I tried to improve the stability of the openQA test module "vlc" with https://progress.opensuse.org/issues/42671 . If you see any further instabilities or random failures please let me know, e.g. in the ticket. I really assume it to be rock-solid stable now :)
Updated by okurz about 5 years ago
- Related to action #47132: [opensuse][u][sporadic] test fails in vlc - vlc does not close added